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SINGULAR CONTROL WITH STATE CONSTRAINTS ON 
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We study a class ol stochastic control problems where a cost of 
the form 

(0.1) E / e-'^''[£{Xs)ds + h{Y:)d\Y\s] 

J[0,oo) 

is to be minimized over control processes Y whose increments take 
values in a cone Y of R'', keeping the state process X = x + B + 
GY in a cone X of R*' , k <p. Here, a; G X, B is a Brownian motion 
with drift b and covariance E, G is a fixed matrix, and Y° is the 
Radon-Nikodym derivative dY/d\Y\. Let C = -(l/2)trace(SD^) - 
b ■ D where D denotes the gradient. Solutions to the corresponding 
dynamic programming PDE, 

(0.2) [(£ + /3)/_^]v sup [-GyDf-h{y)] = 0, 

!/GY:|Gh| = 1 

on X° are considered with a polynomial growth condition and are 
required to be supersolution up to the boundary (corresponding to a 
"state constraint" boundary condition on 9X). Under suitable con- 
ditions on the problem data, including continuity and nonnegativ- 
ity of £ and h, and polynomial growth of i, our main result is the 
unique viscosity-sense solvability of the PDE by the control prob- 
lem's value function in appropriate classes of functions. In some cases 
where uniqueness generally fails to hold in the class of functions that 
grow at most polynomially (e.g., when h — 0), our methods provide 
uniqueness within the class of functions that, in addition, have com- 
pact level sets. The results are new even in the following special cases: 
(1) The one-dimensional case k — p = 1, X = Y = R+; (2) The first- 
order case E = 0; (3) The case where £ and h are linear. The proofs 
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combine probabilistic arguments and viscosity solution methods. Our 
framework covers a wide range of diffusion control problems that arise 
from queueing networks in heavy traffic. 

1. Introduction. This paper is concerned with a class of singular stochas- 
tic control problems with state constraints. The controlled diffusion process 
takes values in a closed convex cone X. The cost is of the form (0.1), and 
the running cost, £, is not assumed to be bounded. The corresponding dy- 
namic programming equation (0.2) is considered with a polynomial growth 
condition, and the role of a boundary condition on 9X is played by the 
requirement that the solution be a supersolution up to the boundary. It is 
well known that classical solutions to Hamilton- J acohi-Bellman (IIJB)-type 
equations do not exist in general, and that an appropriate partial differen- 
tial equations (PDE) framework is via the notion of viscosity solutions (cf. 
Section 2). Our main result is the characterization of the value function for 
the problem as the unique viscosity-sense solution of (0.2). 

For an introduction to viscosity solutions and a list of relevant litera- 
ture, the reader is referred to Crandall, Ishii and Lions [8]. Much of the 
motivation and first examples of this theory came from problems in optimal 
control. See [1] and [15] for accounts on viscosity solutions in deterministic 
and stochastic control and differential games. Control problems with state 
constraint were studied in the viscosity solution framework in the monograph 
by Lions [27] for the deterministic (first-order) case. The relation between 
control with state constraint and viscosity supersolutions of the correspond- 
ing PDE on the boundary was first observed and developed by Soner [32]. 
It was extended to more general first-order equations by Capuzzo-Dolcetta 
and Lions [6]. For stochastic control with state constraints (and no singular 
term), see [25, 26], the recent work of Ishii and Loreti [22] and references 
cited therein. See also [10] and references therein for models in mathematical 
finance with state constraints. 

Many authors have contributed to the study of singular control of diffu- 
sions. One-dimensional problems were studied by direct analysis by Benes, 
Shepp and Witsenhausen [2], Karatzas [23] and Harrison and Taksar [18]. 
HJB-type PDE associated with singular stochastic control were studied for 
their classical and weak (a. e. -sense) solutions by Evans [14], Menaldi and 
Robin [29], Chow, Menaldi and Robin [7] and Ishii and Koike [21]. Contribu- 
tions to viscosity solutions for such HJB equations include [15, 28] and [31]. 
In [28] the authors consider a two-dimensional model that arises from the 
heavy traffic analysis of a queueing network and corresponds to a problem of 
the type studied in the current paper (with linear (. and /i = 0). The authors 
use the viscosity solution framework to establish asymptotic optimality of 
proposed control schemes. The model studied in [31] arises from a problem 
in mathematical finance. The role of the singular control there is, in a sense. 
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antipodal to its role in the current model: it is possible to force the state 
process to move to the origin and remain there, whereas in the current model 
it will be seen that the control can only contribute to a motion away from 
the origin [in the sense of equation (2.2)]. 

Some of the papers that address uniqueness of solutions to HJB equations 
on unbounded domains are as follows. The paper [29] mentioned above in- 
vestigates a class of singular control problems (without state constraints and 
with ^ = 0) and gives various results on characterization of the value function 
as the maximal or the unique solution of the associated PDE in a suitable 
weak sense; however, viscosity solutions are not considered there. In [10] 
the authors study a drift control problem with state constraints and prove 
uniqueness within a class of concave functions. Results for a broad family of 
second-order degenerate elliptic PDE appear in [20] and [8] ; however the re- 
sults on unbounded domains therein do not cover PDEs associated with sin- 
gular control or state constraints. Crandall and Lions [9] consider first-order 
equations on unbounded domain and, motivated by problems in optimal 
control, extend previous results on uniqueness within uniformly continuous 
functions (see references therein) to continuous sub-exponential functions. 
All of the results mentioned above fail to cover PDEs of the form (0.2). 
The delicate nature of the uniqueness problem in the current setting can 
be seen from several simple one-dimensional examples, that are provided in 
Section 2, where uniqueness fails. In particular, when h = 0, uniqueness may 
fail to hold in the class of functions that grow at most polynomially. In such 
cases, our approach establishes uniqueness within the class of functions that, 
in addition, have compact level sets. 

A primary motivation for the problems considered in this paper comes 
from controlled queueing systems in heavy traffic. A formal diffusion approx- 
imation of such systems leads to a class of problems referred to as Brownian 
control problems (BCPs) (cf. [16, 17]). These, in turn can be transformed, 
using techniques introduced in [19], to singular control problems with state 
constraints, of the form considered in this paper. We will demonstrate that 
our result on the PDE characterization of the value function covers control 
problems arising from a broad family of stochastic networks (see Section 3). 

We now remark on some of the key steps in the proof of our main re- 
sult. There are two natural ways to define a value function for the prob- 
lem: A strong formulation, in which infimum of the cost is taken over con- 
trol processes Y adapted to the Brownian motion, and a weak formulation, 
where the infimum ranges over all filtered probability spaces and all control 
processes adapted to the underlying filtration. Denoting the two resulting 
value functions by V and V, it is clear that V <V [see Section 2, equa- 
tions (2.12) and (2.13) for precise definitions]. Solvability of the PDE by 
both V and V is established by means of two different dynamic program- 
ming principles (DPPs) (Propositions 5.1 and 5.2). While the DPP associ- 
ated with V is quite standard and is essentially a consequence of the strong 
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Markov property of Brownian motion, the DPP for V is less straightforward. 
The latter relies on a representation of V as the infimum of the cost over 
controls that are, in an appropriate sense, of feedback form (cf. Section 5). 
Results of this type are well understood for absolutely continuous controls 
(cf. [4]), and go back to a fundamental martingale representation result due 
to Wong [33]. However, in presence of singular control and state constraint, 
our result on the DPP for V appears to be new. The proof also makes use 
of the so-called Skorohod problem to account for the state constraint. For a 
DPP for singular control problems (without state constraints), see [29]. 

The proof of uniqueness of solutions is carried out first for a mixed Dirich- 
let state-constraint boundary value problem on a bounded domain, and then 
lifted to the unbounded domain. Uniqueness on bounded domain uses tools 
from the theory of viscosity solutions, and although several key ingredients in 
the argument have been well developed in the literature, it appears that this 
paper is the first to prove uniqueness of viscosity solutions of the HJB equa- 
tion for a stochastic singular control problem with state constraints. This 
treatment could, in fact, be carried out for a much more general second- 
order degenerate elliptic operator than C The limitation put on the opera- 
tor comes from our treatment of the problem on unbounded domain, where 
certain estimates on the dynamics are used crucially [viz., (4.10)-(4.12)]. 

As already mentioned, the literature on second-order degenerate elliptic 
PDE on unbounded domains fails to capture uniqueness for equation (0.2). 
Our approach uses a verification argument that compares an arbitrary so- 
lution u to the value function V. More precisely, it is first shown that u 
solves a singular control problem on a bounded domain, with an exit cost 
equal to its value on the boundary, giving a variational representation for u 
similar to a DPP (cf. Proposition 6.1). It is here that the uniqueness result 
on bounded domain is required. This sets the ground for comparing V with 
u by means of constructing an admissible control for one problem using the 
other and considering the control problem for u on an increasing sequence 
of domains. In such a construction, a large time sub-exponential estimate 
on the controlled process is crucially used in obtaining the inequality u<V. 
For the inequality V <u, a, control process for the problem associated with 
V is constructed by suitably patching together a sequence of controls for the 
bounded domain problems. 

The paper is organized as follows. Some notation is introduced at the end 
of this section. Section 2 introduces the control problem setting, the PDE, 
the main result and some examples of nonuniqueness. Section 3 demonstrates 
the applicability of the main result to problems that arise from queues in 
heavy traffic. Section 4 contains the bounded domain problem formulation 
and some preliminary lemmas. Section 5 proves solvability of the PDE by the 
value functions, based on the DPPs. Sections 6 and 7 establish uniqueness 
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on bounded, and respectively, unbounded domain. In Section 8, the DPPs 
are proved. Finally, some auxiliary results are provided in the Appendix. 

The following notation will be used. 

For a G M", |a| denotes the Euclidean norm. 

For a set 5 C M", C^iS) denotes the space of twice continuously differ- 
entiable functions on S. C^iS^ denotes the class of continuous functions / 
on S for which all level sets {x G 5" : /(x) < r}, r G R, are compact. Cpoi(5') 
denotes the class of continuous functions f on S for which there is a con- 
stant a = a{f) such that < a(l + x € S. Cb{S) denotes the class 
of continuous and bounded functions on S. C+{S) denotes the class of non- 
negative continuous functions on S, Cpoi,+ = Cpoi H C+, Cb,+ = Cb fl C+ and 
^poi,+ = C'poi,+ n C^. 

For a function /: [0,cx)) — > M", write |/|^ = sup^^jo,*] \f\t for 

the total variation of / over [0, t] with respect to the Euclidean norm. For a 
process X, we use X{t) and Xt interchangeably. 

Denote B^{x) = {y G M'^ : |x — yj < e}. S'^~^ denotes the unit sphere in 
R*^. Infimum over an empty set is regarded as oo. c, ci, C2, . . . denote positive 
deterministic constants whose values may change from the proof of one result 
to another. 

A function from [0, oo) to some metric space E is RCLL if it is right- 
continuous on [0, oo) and has left limits on (0, oo). A process is RCLL if, with 
probability one, its sample paths are RCLL. If ^ is RCLL, denote A^(t) = 
C(0 ~ ^{^~) for t > [see Section 2 for a convention regarding A^(0)]. 

2. Setting and main result. A filtered probability space $ = {^,J-, (J't), 
IP, S), satisfying the usual hypotheses, endowed with a fc-dimensional 
(.Ff)-Brownian motion B with drift b and covariance S is said to be a system. 
Denote by (J^t) the P-completion of the filtration generated by B. We say 
that C is a cone of R*^ if C C M.'^, and if c G C imphes ac G C, for all a > 0. 
We consider a control problem in which a p-dimensional control process Y, 
whose increments take values in a cone Y (in a sense made precise below), 
keeps a fc-dimensional process X{t) = x + B{t) + GY(t) in a cone X, where 
G is a fixed k x p matrix of rank k (k <p). The fc-dimensional cone GY is 
denoted by U. Our precise assumptions on the cones and related notation 
are described in what follows. X (resp., Y, U) is a closed convex cone of R'^ 
[R^, R'"'] with nonempty interior. It is assumed that 

(2.1) unxv^- 

We remark that, unless T, is degenerate, the above condition is necessary 
to guarantee the existence of controls; nonetheless, (2.1) will be assumed 
even for degenerate S. Since U has nonempty interior, (2.1) implies that 
there exists a unit vector uq G UTl X°. Pick yQ (^Y for which GyQ = uq. The 
unit vector uq and the nonzero vector yo will be fixed throughout. Assume, 
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moreover, that there exist a unit vector ui € M'^, a unit vector yi G M^, and 
a constant oq > such that 

u-ui>aQ\u\, u G U, x-ui>aQ\x\, x G X, 

(2.2) 

y-yi>ao\y\, yeY. 

A function y: [0,oo) — > is said to have increments in Y if y{0) G Y and 
y(t) — y{s) gY, 0<s<t<cx3. A process is said to have increments in Y if, 
with probabihty one, its sample paths have increments in Y. 

Definition 2.1 (Admissible control). An admissible control Y for the 
system <^ and the initial data x G X is an (.Ft)-adapted RCLL process with 
increments in Y for which the process 

(2.3) X{t)=x + B{t) + GY{t), t>0, 
satisfies X{t) G X, t > 0, F-a.s. 

By convention, Y{0—) =0 and X{0—) =x. The pair {X,Y) (resp., the 
process X) is referred to as an admissible pair (controlled process associated 
with Y) for $ and x. The class of admissible controls for $ and x is denoted 
by A{^,x), and the class of (^()-adapted admissible controls is denoted by 
A{^,x). When there is no confusion, we refer to A{^,x) [resp., ^(<I>,x)] as 
Aix) [A{x)]. 

Before introducing the cost functional, we fix some notation. Associated 
with a nondecr easing function (f):[0,oo) ^ M+, we define a cr-finite mea- 
sure m on ([0, oo), S([0, oo))) via the relations m{a,b] = (l){b) — 4){a) for 
a, 6 G (0, oo) and m{0} = (/)(0). For '0 G ^^[0, oo), S([0, oo)) , m) , we will write 
/[o oo) ^('^) d4>{s) = /[Q ip{s) dm{s). Note that with this notation, the Stielt- 

jes integral j^'4){s) d(t){s) is given as t] '^(•s) c^0(s)- 

Let y : [0, oo) — s- Y be an RCLL function with increments in Y, and note 
that, by (2.2), it has bounded variation over finite intervals. Then it can be 
written as 

(2.4) y{t)=f y°{s)d\y\{s), 

J[o,t] 

where y°, the Radon-Nikodym derivative dy/d\y\, is a measurable function 
with values in Y fl 5^"^ (see Lemma A.l in the Appendix). Let /i : Y ^ R be 
a continuous function satisfying the radial homogeneity condition 

(2.5) h{ar]) = ah{r]), a>0,r?GY. 
Denote 

(2.6) / f{s)h{dy{s))= I f{s)h[y°{s))d\y\{s), 

J[a,b] J[a,b] 
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for every function / for which the right-hand side is well defined. The no- 
tation y° and / fh{dy) of (2.4) and (2.6) is used throughout. In a similar 
manner, given a system $ and an {.T^^j-adapted RCLL process Y with incre- 
ments in Y, we can find an {^^-progressively measurable process with 
values in YnS^'^ such that Y{t) = J^^^ Y°d\Y\{s) (and clearly the value of 
the integral is independent of the choice of such {5^°}). For this statement, 
see Lemma A.l. Once more, we will abbreviate /[^ 6] /(s)/i(y°(s)) d|y |(s) 
by /j^ f{s)h{dY{s)). The cost associated with given system initial data 
rr € X and admissible pair {X, Y) is given as 

(2.7) J($,x,y) = E/ e'^'[£iXs)ds + hidYs)], 

J[0,oo) 

where, here and throughout, E denotes expectation with respect to P, and 
/3 > is a constant. 

Remark 2.1. (a) In order to formulate the control problem, one needs 
to define h only on YriSP~^. However, the radial homogeneous extension of 
such h to all of Y will turn out to be convenient. 

(b) In the special case where h is linear, say, h{y) = ho ■ y, the integral 
in (2.6) is the same as / /(s)/io • dy{s). 

(c) The definition (2.6) reflects the formal identity, h{dy/d\y\) d\y\ = h{dy), 
suggested by the radial homogeneity of h. Of course, the notation / fh{dy) 
should not be confused with a Lebesgue integral against a measure h. 

Following are our assumptions on i and h, the function i is in C4.(X) and 
there exist constants Q,i,Q^2,Q,3 G (0, cxd), mi G [0, oo) such that 

(2.8) Ci,i\xr-Ci,2<iix)<Ci,3{\xr + l), x£X. 

Note that mg = corresponds to the case that i is bounded. Let mod(r, 6) = 
sup{^(x) — £{y) : X, y G X n i?r(0) : |x — y | < 5} denote the modulus of conti- 
nuity of i on Br{0). It is assumed that 

(2.9) mod{r + l,6)<m{5){l + r"''^), <5>0,r>0, 

where f?T,(0-|-) = 0. Note that (2.9) is clearly satisfied if £ is a polynomial. 
In addition to (2.5), the function h is assumed to be (globally) Lipschitz, 
convex, and nonnegative on Y. All assumptions mentioned thus far apply 
throughout this paper. At several places we will also use the following condi- 
tions, under which sharper results will be obtained. We will explicitly refer 
to them when they apply: 

(2.10) Either S is nondegenerate or ui • 6 > 0; 

(2.11) h{y)>Ch\y\, y G Y for some constant c/j > 0. 



8 R. ATAR AND A. BUDHIRAJA 

We will consider two notions of value function for the control problem. 
Let 

(2.12) F(x)=inf inf J(^,x,Y), 

* YeA{'S>,x) 

where in the outer infimum <1> ranges over all systems, and let 

(2.13) V{x)= inf J{^,x,Y). 

Given any two systems $ and for every Y € A{^,x), one can find Y G 
A{^,x) such that {B,Y) is equal in law to (B,Y), and thus V does not 
depend on 

Consider now the equation 

(2.14) {{C + f3)i;-i)vn{Dij) = 0, 
where C denotes the differential operator 

(2.15) £ = -itrace(SD2) -6-L>, 
and 

(2.16) n{p) = sup -{Gyp + h{y)), p€R\ 
where Yi = {y G Y: |Gy| = 1}. 

Definition 2.2 (Constrained viscosity solution), (i) For 5 C X, a con- 
tinuous function ip : S ^ [0, oo) is said to be a viscosity supersolution (resp., 
subsolution) of (2.14) on S if for all x S S" and all ip € C'^{S) for which ijj — f 
has a global minimum (maximum) at x one has 

{I3^{x) + Cifix) - £{x)) V n{Dip{x)) > (< 0). 

(ii) •0 is said to be a constrained viscosity solution of (2.14) on X if it is a 
viscosity supersolution of (2.14) on X and a viscosity subsolution of (2.14) 
on X°. 

Our main result characterizes the value function as a constrained viscosity 
solution of (2.14). 

Theorem 2.1. If rn^ > {resp., = 0) then the functions V and 
V are m C^„i,+(X) {resp., Cy^^[X)\. 

(ii) Solvability. V and V are constrained viscosity solutions of (2.14) on 

X. 

(iii) Uniqueness. Let (2.10) hold. Then in the case that mi > 0, V is the 
only such solution in the class Cp^j _|_(X); and in case that m£ = 0, V is the 
maximal solution in the class Cpoi,+(X). If, in addition, (2.11) holds, then 
uniqueness holds in Cpoi,+ (X), irii > 0. 
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(iv) // (2.10) holds, then V = V. 

Remark 2.2. Example (a) below shows that condition (2.11) in part 
(iii) of the theorem is, in a sense, necessary. Example (b) demonstrates the 
role of the function class C^^i + ■ X = Y = , G = 1, P = 1, ui = 1 and 



(a) Condition (2.11) cannot be dropped in general. Consider the data 
6 = 0, S = 1 and £ = 1. Then (2.10) holds and (2.11) fails. Clearly = and 
V = 1 on M_|_. In this case the statement regarding uniqueness in Cpoi,+ fails 
to hold, as one checks that, for every c G [0, 1], = c is a constrained viscosity 
solution of the equation on M+. This example also demonstrates that in 
the statement regarding = 0, one cannot in general replace "maximal 
solution" by "unique solution." 

(b) The conclusion ^^uniqueness in C^^i ^" cannot, in general, be replaced 
by '''' uniqueness in Cpoi,+ ." Assuming now 6=1, = O5 /(x) = x + 1, one 
finds that V{x) = x + 2. Also, it can be easily checked that for every c € [0, 1], 

= c is a solution. Hence, it follows that, under (2.10) alone, uniqueness in 
Cpoi,+ does not hold in general. 

Remark 2.3. If £ is convex and h is linear, the equality of V and V is 
an immediate consequence of Jensen's inequality. 

3. Stochastic networks. In this section we discuss applications of our re- 
sult to BCPs. The formulation introduced here is used only in this section, 
and readers who are not interested in this aspect can safely skip it with- 
out losing continuity. As mentioned in the Introduction, BCPs arise from 
queueing control problems considered in their formal diffusion limit and 
they often can be transformed into singular control problems of the form 
studied in this paper. The transformed control problem is sometimes re- 
ferred to in the literature as the equivalent workload problem. Our objective 
in this section is to describe how our results apply to equivalent workload 
problems corresponding to a broad family of stochastic networks. To this 
end, we first define BCPs and quote results of [19] regarding reduction to 
a singular control problem (no attempt is made to discuss the underlying 
queueing model or how the BCP arises from it). BCPs were introduced by 
Harrison in the important work [16] (see [17] for more general formulation). 
Our presentation follows [19]. Let 




(3.1) 
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where (il, JF, (^(),P) is a filtered probability space satisfying the usual hy- 
pothesis, and {B{t);t > 0) is an m-dimensional (.7^t)-Brownian motion, with 
drift b and covariance S. The problem data of a BCP is an m x n matrix 
R, a p X n matrix K and a vector z S (termed input-output matrix, 
capacity consumption matrix and, resp. initial inventory vector). We follow 
the notation of [19] as far as dimensions of vectors and matrices are con- 
cerned, except that we use the symbol k in place of [19] 's d. The matrix K 
is assumed to have rank p {p<n). 

Definition 3.1 [Admissible control for BCP). An admissible control 
{L{t)]t> 0} for the BCP, associated with $ and z G W^, is an ROLL (Tt)- 
adapted process with values in such that, setting Z{t) = z + B{t) + RL{t), 
t > 0, and Y{t) = KL{t), one has that Z{t) € M'" for ah t > 0, and Y has 
increments in M.^. 

Denote by A{^,z) the class of all admissible controls for the BCP asso- 
ciated with ^ and z. The goal is to minimize 

(3.2) J{^,z,L) = E f e-'^*[i{Z{t))dt + h-dY{t)], 

J[0,oo) 

where 7GC+(M!f^), and /iGM^. Let V{z) =mf~mf^^j^~ ^^J{^,z,L). 

Let B = {X ■.KX = 0}, let M = RB CM."", and let q be the dimension 
of Af. The dimension of = A/""*" is then k = m — q. Let M be any k x m 
matrix whose rows span M. By Proposition 2 of [19], there exists a k x p 
matrix G such that MR = GK. The choice of G, in general, is not unique. 
Set X = MM™, Y = and U = GY, and note that both X and U are subsets 
of M'^. Define £:X^M+ as 

l{x) = -mi{i{z) ■.ze^+,Mz = x}. 

Assume 

there exists a continuous function g:\^ [0,oo) such 

(3.3) that g{x) e argmin{£(2;) : z G and Mz = x}, for all 

xex. 

With the data X, Y, G, U = GY, S = MT,M^ ,_b = Mb, J, h, the singular 
control problem of Section 2, and in particular, A{x) and V are well defined. 

Theorem 3.1 ([19]). Given zeW^, letx = Mz. Then V{z) =V{x). 

In fact, Harrison and Van Mieghem [19] give an explicit way of construct- 
ing an L from a Y such that J{^,z,L) = J{^,x,Y) (where ^ consists of 
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the same filtered probability space as ^ and is equipped with the Brownian 
motion B). 

We now list some sufficient conditions for our characterization results to 
hold. For example, if i is linear, nonnegative on W^, and vanishes only at 
zero, conditions (2.8) and (2.9) are satisfied (in fact, £ is piecewise linear and 
m£ = 1). From Theorem 2 of [3], it follows that (3.3) holds as well. Next, 
if G and M have full rank, then X and U have nonempty interior as sub- 
sets of R'^, as required. In, case M and G have nonnegative entries, (2.2) 
is satisfied with any fixed unit vectors ui and yi in the respective positive 
orthant. Finally, let us assume that S is nondegenerate. These assumptions 
hold for a broad family of stochastic networks. Under the heavy traffic con- 
dition (cf. Assumption 1 of [17]), one can choose G with nonnegative entries 
(see [17], equation (3.12)). Conditions for a nonnegative choice for M have 
been given in Theorem 7.3 of [5]. In particular, these conditions hold for open 
multiclass queueing networks (cf. [5], Section 3.1), parallel server networks 
(cf. [5], Section 3.2) and several other classes of unitary networks (see [5], 
Corollary 7.4). As a result, these families of networks are covered under our 
characterization results. 



4. Preliminary results. In this section we study some basic properties of 
the value functions V and V [cf. (2.12), (2.13], as well as those of the value 
function of an analogous problem on a bounded domain, defined below. For 
r > 0, denote 



(4.1) 



Xf. = {x S X : x • Si < r}, dr = {x x ■ ui = r}. 



We will always write X^ for X\Xr. By (2.2), X^ and dr are bounded sets. 
Fix a system and let A{x) =A{^,x). For Y G A{x), let X = x + B + GY 
be the corresponding controlled process, and set cr = cr{r) as 



(4.2) 

For x G Xr , let 
(4.3) Ar{x) 



a = mi{t:Xt^Xr}. 



{¥ G A{x) : on the set {a < oo},Xo- G dr}. 



Let (j) be any function in C+{dr)- Define for x G X^ and Y G Ar{x) the cost 
for the bounded domain problem Jr{x,Y) = Jr^{x,Y) as 



(4.4) Jr,^{x,Y)=E 



e-f^'[i{X,)ds + h{dYs)] + ^(X^) 



[0,a] 



where here and throughout we use the convention that, on the event {a ■ 
oo}, [0, cr] = [0, oo), and e~^" f{X(j) = 0. Let also Vr = Vr^,p be defined as 

(4.5) Vr{x)= inf Jr{x,Y), x^%. 

Y&Ar{x) 
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The notation, Ar{^,x), Jr{^,x,Y) and Vr{x) is used analogously to x), 
J{^,x,Y) and V{x) defined in Section 2. 

We first state and prove a result related to the Skorohod problem [11, 
12]. For E= [0,oo) oi E = [0,T], let V{E:R'') denote the space of RCLL 
functions from E to M^. Denote Px([0, oo) : M'=) = {z e V{[0, oo) : M'^) : ^(0) € 
X}. Define Px([0, T] : M*^) analogously. 

Lemma 4.1. There exist maps F from Dx([0,oo) -.M.^) into itself and T 
from Px([0, oo) :Mf') into D([0, oo),M) such that the following hold: 

(i) If z eVx{[0,oo) :R''), v = f{z) andx = T{z) then v : [0,oo) ^ R+ 
is RCLL and nondecreasing, x = z + uqv = z + Gy^v, and x{t) G X for all 
t>0. 

(ii) The maps T and T are Lipschitz in the following sense. For all 
zi,Z2eVxi[0,oo) ■.R'') and allT>0, 

\f{zi) - f{z2)\*T + |r(zi) - r(z2)|T < - Z2\t, 

where the constant k <oo does not depend on T, z\ and z^. 

(iii) The map F is nonanticipating in the following sense. For every T G 
(0,oo) there exists a map Ft : X'x([0, T] : M'^) ^V{[0,T]:R), such that for 
zeVx{[0,(^):R''), 

(iv) // z e !Dx([0, oo) :M'^) takes values mX on [0,T], then v = T{z) van- 
ishes on [0,T]. 

The notation F and F is kept throughout this paper. 

Remark 4.1. Clearly, T{z) = z + Gyf)T{z). The lemma will help us con- 
struct one admissible control from another, as follows, li X = x + B -\- GY is 
the controlled process for a y G A{x)., then letting Q = F(x + B + GY), the 
process Y = Y -\- y^C, is seen to be admissible for x. 

Proof of Lemma 4.1. Let 5 be the collection of ah unit vectors s G M'^ 
such that G M'^ : s • ^ > 0} D X. Then one has the following representation 
for X (cf. [30]): 

X = f|{eGM^s-C>0}. 

s&S 

Recall that {to G X°. For ^ G M'^, let 7r(^) denote the projection of ^ onto the 
boundary ffK along uq. Explicitly, 7r(^) = ^ + a{Quo, where 

q(^) = sup — ^ • s / uo ■ 
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It is elementary to check that the range of vr is (9X and that vr is globally 
Lipschitz. Given z G PxliO, oo) :]R'=), let 

v{t) = V sup So • (vr(z(s)) - z{s)), t > 0, 

0<s<t 

and X = z + uqv. Then using the fact that Tr{z{t)) = 7r(x(t)), it is not hard to 
check that uq • {x{t) — 7r{x{t))) > 0, t>0. Thus, x{t) = 7r{x{t)) — a{x{t))uo, 
where a{x{t)) < 0. Since both mq and 7r(x{t)) are in X, this shows that 
x(t) G X, and part (i) of the lemma is established. Parts (ii), (iii) and (iv) 
follow by construction and the fact [used in proof of (ii)] that vr is globally 
Lipschitz. □ 

Recall that h is radially homogeneous and convex. This is easily seen to 
imply that 

(4.6) hiy + z)<h{y) + h{z), y,zeY. 
As a result we have the following. 

Lemma 4.2. Let Yi and Y2 have increments in Y and set Y = Yi + Y2. 
Then for all t G (0, 00), 

(4.7) / e-f^'h{dY{s))< f e-^'h{dYi{s))+ f e-'^'h{dY2{s)). 

J[0,t] J[0,t] J[0,t] 

Let, for i = 1,2, /ij be a a-finite measure on (]R_|_, ;B(M+)) defined as 
lii{B)= [ d\Yi\{s), BeB{R+), 

JB 

where B{M.j^) is the Borel a-field on M4.. If and ^2 o-fe mutually singular, 
then (4.7) holds with equality. 

Proof. Along the lines of [15], page 320, let Y{s) = yi ■ Y{s) and denote 
dC{s) = d\Y\{s) and di{s) = dY{s). Also let 

fi{B) = f de, MB) = I dl 

JB JB 

for Borel B C [0, 00), and define similarly Yi, d^i, d£^i and Jli for f = 1,2. 
By (2.2), ao|y| < yi • y < |y| for y G Y. Hence, /i and fi are mutually abso- 
lutely continuous, and a similar statement holds for and /zj, i = 1,2. Since 
Y = Yi + Y2 and Yi are nondecreasing, clearly, /2j is absolutely continuous 
with respect to Ji, for i = 1,2. This shows that /ij is absolutely continuous 
with respect to fi, and we denote by d^i/d^ the respective Radon-Nikodym 
derivatives. Thus, 

^ di^^^ dC ^ 
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By (4.6), 

(4.8) h{Y°{s)) < Y: ^{s)h{Y°{s)), /z-a.e. 
and as a result 

(4.9) / e-^^h{Y'^{s))das)<J2 [ e-P^h{Y°{s))dUs). 

Next, suppose that and ^2 are mutually singular. Then there are disjoint 
sets Si E ;S(M+) such that fJ-iiSf) = 0. Clearly, dS^i/dS, = Is'-d^j/d^, //-a.e.. 
Since (4.6) holds with equality whenever either y or z vanishes, (4.8) holds 
with equality, and so does (4.9). □ 

Lemma 4.3. Let x eX and y G Y. If x + Gy £X, then 

Vix + Gy) + h{y) > V{x), V{x + Gy) + h{y) > V{x). 

Proof. We will only prove the result for V . The proof for V is similar. 
Assume without loss of generality that A{x + Gy) is nonempty. Let Y € 
A{x + Gy) . Then the corresponding controlled process X = x + Gy + B + GY 
takes values in X. Set Y = y + Y and X = x + B + GY. Clearly, X = X , and 
Y has increments in Y and, therefore, Y € A{x). Also, AY{0) = y + AY{0), 
and therefore, by (4.6), /i(AF(0)) < h{y) + h{AY{0)). Thus, 

J{x + Gy,Y) > J{x,Y) - h{y) > V{x) - h{y). 

Since Y G A{x + Gy) is arbitrary, the result follows. □ 

In the proof of the next result, and several times in the paper, we use 
the fact that if X is the controlled process corresponding to some Y G A{x)., 
then, by (2.2), for < t < s < 00, 

(4.10) \Xs\ >ui-Xs>ui-Xt + ui- {Bs - Bt) > ao\Xt\ - \Bs - Bt\. 
As immediate consequences of this inequality, we have 

(4.11) 2™E|A:,r>a[}'E|Xtr-2'"E|5t-B,r, s>t,m>0, 

(4.12) \X\t<2a^\\Xs\ + \B\:), s>0. 

Lemma 4.4. There exist constants 01,02,03 > such that 
oi|x|™« - 02 < V{x) < 03(1 + \xr'), x G X. 
The above inequality also holds with V replaced by V . 
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Proof. Once more we will only prove the result for V. Let x S X and 
define Y = yof{x + B) and X = T{x + B). Then X = x + B + GY , and it 
follows from parts (i) and (iii) of Lemma 4.1 that Y G A{x). 

Denoting by the zero trajectory in M'', it is clear that T{x + 0)(t) = 
x, t>Q. Hence, by Lemma 4.1(ii), \X{t) - x\ + \Y{t)\ < ci\B\^. With the 
notation of the proof of Lemma 4.2, the Radon-Nikodym derivative of // 
w.r.t. p, is bounded above by QjQ , 9/S follows from (2.2). Thus, 

/ e-^'d\Y\{s)<aQ^ f e-'^'dY{s) 

J[0,oo) J[0,oo) 

(4.13) ={ao(3y^ f e^^'Y{s)ds 

J[0,oo) 

<{aoPr' I e-P'\Y{s)\ds. 

J[Q,oo) 

By (2.8) and (4.13), 

J(x,y) = E / e~l^\l{X{t))dt + h{Y°{t))d\Y\{t)) 

J[0,oo) 

<C2e/ e~'^\l + \X{t)r' + \Y\{t))dt 

J[0,oo) 

< C3E / e-^*(l + |xr^ + (|5|*)"**^i) dt 

<c4(i+ixro, 

where 02,03,04 do not depend on x. By (2.8) and (4.11), for every admissible 
J(x,y)>C5E / e-'^'lXtr^t-ce >C7|2;r^ -C8, 

J[0,oo) 

where 07,03 > are independent of x and Y . □ 
Lemma 4.5. V and V are continuous on X. 

Proof. We will only consider V . Fix r > 0, and given arbitrary e € 
(0, 1), consider xi,X2 € X n Br{0) with |xi — X2\ < S < I, where 6 > will 
be chosen later. Fix Yi € A{xi) such that 

(4.14) J(xi,yi)<F(xi) + e/2, 

and let Xi be the corresponding controlled process, namely, Xi = xi + B -\- 
GYi. Let Z = f{x2 + B + GYi), = + y^Z and X2 = X2 + S + ^^2 = 
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X2 + B + GYi + uqZ. By Lemma 4.1, I2 £ •^{x2) and X2 is the corresponding 
controlled process. Note that 

Xi = r(Xi), X2 = T{x2-xi+Xi). 

Hence, by Lemma 4.1, 

(4.15) |Xi(t) - X2{t)\ + \Y^{t) - Y2{t)\ < ci5, 

where ci does not depend on t, xi and X2- Assume that 5 is small enough so 
that c\5 < 1. In particular, this shows that /jq e~^* dZ{t) < C26, and thus, 
by (2.8), (2.9), Lemmas 4.2 and 4.4, we have 

J{X2,Y2) - J{xuYi) <E I e-f''mX2{t))-i{Xi{t)))dt + h{m)dZ{t)] 

J[0,oo) 

<E / e~^*mod(|Xi(t)| + l,5)dt + C35 

J[0,oo) 

(4.16) <Cim{ciS)E f e-f^\i{Xi{t)) + 1) dt + C3S 

< C5m{ci5){V{xi) + 1) + C35 

<C6m(ci(5)(r™^ + l) + C35. 

Choosing 6 so small that the expression on the last line is bounded by e/2, 
we conclude that V{x2) — V{xi) < e whenever \xi — X2\ < S and xi,X2 € 
XnS(0,r). Since r is arbitrary, V is continuous on X. □ 

Remark 4.2. Lemmas 4.4 and 4.5 are seen to imply part (i) of Theo- 
rem 2.1. 

For X € X,. and u € U for which there exists p > such that a; + G XJ; , 

let 

(4.17) -frix,v)=mi{p>0:x + pveX^^.}. 
Clearly, x + jr{x,v)v £ dr for x,v as above, and 

(4.18) xeXr,veV, x + vGX^ =^ 7r(x,T;)<l. 

Lemma 4.6. Let cj) G C+{dr) and suppose that (2.10) holds. Then Vr^cj, is 
continuous on X^. 

Proof. Below we use implicitly the fact that, in the definition of Vr, 
the values X and Y take on the interval (cr, 00) are immaterial. Fix e > 
and consider all xi,X2 G X^ with \xi — X2\ < 6 < 1/2, where 6 > will be 
chosen later. Let Yi G Ar{xi) be such that 

(4.19) JrA^i,Yi)<Vr,4xi)+e/2, 
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let Xi be the corresponding controlled process, Xi = X^^ = xi + B + GYi, 
and let ai be the corresponding exit time. Let Z = T{x2 + B + GYi) and 
X = r{x2 + B + GYi). By Lemma 4.1(i), (iii), Z is M+-valued, adapted, 
RCLL and nondecreasing, and X{t) G X, t > 0. Also, by Lemma 4.1(iv), 
T{xi -\- B + GYi) = 0, hence, by Lemma 4.1(ii), 

(4.20) < Z{t) <k6, t> 0. 

Let Y = Yi + VqZ. Define r = va.i{t: X{t) € X^}. Let o"2 = 0"i A r and set 
(X2,F2) = (X,y) on [0,0-2). Define also 

V ; 21 2; \y(cji)+7,(X(cTi),no)yo, c^i < r, 

and 

(4.22) X2{a2)=X2{a2-) + G/\Y2{a2). 

Note that X2(cr2) G dy. Hence, X2 is well defined until cr2, the first time it 
exits X^. We leave X2 undefined on (cj2,oo). Below we sometimes write Ui 
for GYi, i = l,2. 

Note that (T2 < By (4.17) and (4.21), the random variable AX2{cr2) is 
U- valued and .Tvj -measurable, and since 

X2{t)=X2 + B{t) + GY2{t), t<a2, 

we see that Y2 G ^4(^2) and X2 is the corresponding controlled process. On 
the time interval [0,cj2) we have 

Xi = T{Xi), X2 = r(x2-xi+Xi), 

and therefore, by Lemma 4.1, 

(4.23) \Xi{t)-X2{t)\ + \Yi{t)-Y2it)\<ci\xi-X2\<ci6, t < a2, 
and, similarly, 

(4.24) \X^{t)-X{t)\<ci6, t<a2, 

where ci does not depend on t,xi,X2 and 6. 

We show that there exists a constant c not depending on xi,X2 and 5 
such that 

(4.25) 1X1(^2) - X2{a2)\ + hiAY2ia2)) - h{AYi{a2)) < cS. 
Different arguments are used in the two cases below: 



18 R. ATAR AND A. BUDHIRAJA 

Case (i): r < cji. By (4.17)_and (4.21), X{t) - X2{t) £ U. Moreover, 
since X2{t) £ dr and ^i(t) € X^, ui • {X2{t) — Xi{t)) > 0. Therefore, by 
(2.2) and (4.24), 

ao\X{T) - X2{r)\ < ui ■ {X{t) - X2{t)) 

(4.26) 

<ui-{X{t)-Xi{t))<ci5. 
Combining (4.24) and (4.26), we get 

(4.27) \Xi{a2)-X2{a2)\<C25. 

Since X(r-) G and X{t) G X^, it follows from (4.18) that 7 = -fr{X{T-), 
AX{t)) < 1. With (4.21), we have AY2{a2) = 'yAY{a2) = -fAYi{a2) + 
7yoAZ(o"2). Thus, by the Lipschitz property of h and (4.20), 

(4.28) ^^^2(^72)) < /i(7Ayi(c72)) + C37|yo|AZ(a2) < h{AYiia2)) + 0^5, 
and (4.25) follows. 

Case (ii): ai < r. Note that ai = a2 and by (4.24), \Xi{a2) - X{a2) \ < 
ci6. Since Xi(a2),X2{a2) € dr, we have ui ■ Xi{a2) = ui ■ X2{(J2)- Also, note 
that X2(o-2) - X{a2) G X. Thus, using (4.24): 

ao\X2{a2) - X{a2)\ < ui ■ {X2{a2) - X{a2)) 

(4.29) 

= ui ■ {Xi{a2) - X{a2)) <ciS, ai<T. 

Combining (4.24) and (4.29), the estimate (4.27) holds for an appropriate 
constant C2. Now, by (4.21) and (4.22), 

AX2(ai) = GAY2{ai) = G{Yi{ai-) - ^3(^1-)) + GAYi{ai) + YSq, 

where 7' = 7r(X((Ti), uq) + Z{ai). Since AXi{ai) = GAYi{ai), we have 

7' < |AX2((7i) - AXi{ai)\ + C5|yi(ai-) - Y2{a^-)\, 

and since ai = 02, we have, by (4.23) and (4.27) that 

(4.30) 7' < C6(5. 

Using again (4.21), AY2{(^'i) — AY\{cj2) = 7'yo- Combining (4.30) and the 
Lipschitz property of /i, we establish (4.25). 
Let 

m(a) = max{|^(y) — ^(-z)| :y,z £ X^, \y — z\ < a} 

Vmax{|(/)(y) - cl){z)\ : y , z £ dr, \y - z\ < a}. 
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Recalling the convention e~^^ f{X(^) = when a = oo, 

<E f e~''\£{X2{t))-£{X,mdt 
J [0,0-2] 

+ hidYiit)) - h{dYi{t))] 

(4.31) +E{l,,=,,e-'''^2((^(X2(cT2)) -(A(Xi((j2)))} 

+ E{1,,<,, 0(^2 (a2 ) ) - e"^'^^ (t>{Xi{ai))]} 

< cj5 + C8m(c85) 

+ E{l,,<,Je-^'^20(X2(a2)) - e-^"i(/.(Xi(ai))]}. 

The estimate 07^ in the last line above follows on using Lemma 4.2, (4.20) 
and (4.25). By (4.27), since X2{(T2) Gdr,we have r — ui ■Xi{a2) < cg5. Thus, 
for a > 0, (Ji > £72 + a implies that, for t € [o"2, 0-2 + a], 

r >ui- Xi{t) > ui ■ xi + ui ■ B{t) + ui ■ Ui{a2) 

= ui-Xi{a2)+ui-{B{t)-B{a2)) 

>r-cg5 + ui- {B{t)-B{a2)). 

Therefore, 

P(cri > (72 + a) < Pf max ui ■ {B{a2+t) - B{a2)) < cgS] 

\0<t<a / 

(4.32) 

= Xi{a,6). 

Note that, if either S is nondegenerate or • 6 > 0, we have that, for each 
fixed a, Xi{a,6) — >■ as (5 — > 0. Moreover, writing Xi{ai) — Xi{a2) = B{cri) — 
B{a2) + Ui{ai) - C/i(o-2), we have ui ■ {Ui{ai) - Ui{a2)) < cg6 + \B{ai) - 
B{a2)\. Using (2.2), on {0 < cii - a2 < a}, 

|Xi(cJi) - Xi((j2)| < cio<5 + cio max{|5(c72 + t) - 5(fT2)| : t G [0, a]} 

(4.33) 

= cio(5 + A2(a). 
Using (4.32) and (4.33) in (4.31), we obtain 

Jr,<p{^2, U2) - Jr,<j){xi,Ui) 

< cii(<5 + m(c4<5) + Ai(a, 6)+a + E(m(cio<5 + A2(a)))) 

< cii(<5 + m(c4<5) + Ai(a, 6)+a + m{2cioS) + E(m(2A2(a)))). 

Note that E(?n(2A2(a))) ^ as a ^ since m2 is uniformly bounded (bound 
only depends on r) and A2(a) — > as a — > 0. Choose a small enough and 
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then 6 small enough so that the right-hand side is bounded by e/2. Combin- 
ing this with (4.19), we have that yr-,(/)(2;2) < ^.(^(a;!) + e whenever — X2I < 
6. This proves the continuity of Vr^cf, on X^. □ 

The following lemma shows that the infimum in the definition of V{x) [ 
V{x)] in (2.13) [resp. (2.12)] can equivalently be performed over a class of 
admissible controls under which the controlled process's moments are finite 
and sub-exponential in the time variable. 

Lemma 4.7. For x G X and a system <I>, let 

Af{x) = I^YeA{x) : V a,t >0 E|Xy(t)|" <oo; 

V a > lim e-^*E|Xy (t)|" = ol, 

t — >oo J 

where Xy is the controlled process corresponding to Y. Define Af{^,x) 
similarly by replacing A{^,x) above by A{^,x). Then 

(4.34) V{x) = inf{ J(x, F) : G Af{^, x)} 
and 

(4.35) V{x) = mimi{J{x,^,Y):Y e Af{'^,x)}. 

Proof. We wih prove (4.34). The proof of (4.35) is identical. Fix x G X. 
Given p > \x\, an admissible control Y and the corresponding controlled 
process X for which J{x, Y) < 00, let \p = mf{t : ui • X{t) > p}. Note that 
\p may assume the value 0. Let 

Zp{t) = X(Ol[o,Ap)(i) + {X{\p-) + B{t) - B{\p))ly^^^^){t), t > 0, 

where l[o,Ap) is understood as zero in case Ap = and X{0—), by convention, 
equals x. Set 

yp = yi[o,Ap) + yof(Zp), 

(4.36) 

Xp = r{Zp) = x + B + Gyi[o,Ap) + uof (Zp). 

By Lemma 4.1, Yp is an admissible control and Xp is the corresponding 
controlled process. Denote Xp{t) = ^(t)l[o,Ap)(*) + -'^(-^p~)l[Ap,oo)- Clearly, 
Xp takes values in X and, therefore, Xp = T(Xp). Thus, by Lemma 4.1, on 
{Ap < 00}, 

I -^p ^^p I '7^ ^ ^ I ^p -^p I "j^ 

(4.37) 

= Ksup{\B{s) - B{Xp)\: s G [Ap,r]}, T > Ap. 
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Thus, by (4.36), the right-hand side of (4.37) is an upper bound for \Xp{T) — 
Xp{Xp—)\ for T > Xp. A similar argument using the fact that T{Xp) = 
shows that the right-hand side of (4.37) is an upper bound also for \Yp{T) — 
Yp{Xp—)\. Thus, have on {Xp < oo}, 

\Xp{T) - Xp{Xp-)\ + \Yp{T) - Yp{Xp-)\ 

(4.38) 

< ci sup{\B{t) - B{Xp)\ : t G [Xp, T]}, T > Xp. 

Since on {Xp < oo}, we have \Xp{Xp—) \ = \X{Xp—) \ < a^^p, the sub-exponential 
behavior of the ath moment of Xp{t) follows from a similar property of the 
Brownian motion. Thus, to prove (4.34) it suffices to show that 

(4.39) limsup J(x,yp) < J(x,y), YeA{x). 
Note that, by (2.8), 

l[X„oo)isMXis)r<liXs)+C,,2 

and since J{x,Y) < oo, we have E e~^^{£{Xs) + 01^2) ds < 00. Also, 
clearly, Ap — > 00, a.s., as p ^ 00. Thus, e~^*|X(s)|™'^ (is — > a.s. and 

in L^. Using (4.10) with t = Xp—, this shows that 

(4.40) jnn E[e-^^''|X(Ap-)r^] = 0. 

Note that, with ^ = yof (^p), [Yp{t)-Yp{Xp-)]lt>x^ = ((t). Hence, using (4.38), 
we get 

e-^'h{di{s))<c ( e-l^' d\i\{s) 

[Ap,oo) v/[Ap,oo) 



<c( e-P'\Yp{t) -Yp{Xp-)\ds. 

Therefore, with (2.8) and (4.38), we have 
J{x,Yp)-J{x,Y) 

<Q,3E / e-l''[{\Xp{s)r + l)ds + h{di{s))] 

J[Xp,oo) 

<C2e[ e-'''(\XpiXp-)r+ sup \Bs-Bx,r''^ + l)ds, 
"'[Ap.oo) \ te[Xp,s] / 

where the constant C2 does not depend on p and Y . Using the above along 
with (4.40) yields (4.39). This proves the lemma. □ 

The following lemma shows that, in computing the value function, the 
class of admissible controls can be further restricted. 
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Lemma 4.8. Fix x £X, a system $ and Y £ Af{^, x) with J{x, <I>, Y) < 
oo. Then one can find a c G (0, oo) such that, for all £ G (0, 1), setting 



(4.41) T^ = mf(i:E/ e-^"(^(X,) + /i(dy,)) < el V log(e-^/^) 

I J[t,c<,) ) 

(4.42) 



[t,oo) 

Z'{t)=X{t)l[o,T4t) + iX{T') + B{t) - i?(T^))l(T^oo)(t), 

t>0, 



and 



(4.43) 



y^ = yi[o,T^] + yor(z^), 

X' = T{Z') = x + B + GFl[o,T-] + uoT{Z'), 
we have that, for all T >T^ , 

(4.44) \x^- XTA + \yT-YTA< csup{|5(t) - s(r^)| :t € [^^^]} 

and J(a;, y=) < J(x, F) + ce. 

Proof. Note that X^ is the controlled process corresponding to Y^ . 
Since X'^it) G X for t < we have f (Z^)(t) = for t < and so 

(4.45) {Y^{t),X^{t)) = {Y{t),X{t)), t<T^. 

Arguing again by Lemma 4.1, letting X^ = X{- A T^), we have X'^ = r(X^) 
and therefore, for T >T^ , 

IX" -X^\t< k\Z^ - ^"It = fisup{\B{t) - B{T'')\:te [T^^]}. 

Combining the above with (4.43) yields (4.44). 

Next, moment estimates on the Brownian motion imply 

(4.46) E|x^(r)r^ <ci(E|XT.r^ + (T-r^)'^ + i), r>r^ 

where ci,m do not depend on T,£ and Y. Hence, 

(4.47) E / e-'^'\X'{s)r' ds < cse"^^' (E|Xt. 1™^ + 1). 

J{T-,oo) 

Arguing as in the previous lemma, by (4.44), we also have 



E / e-^"/i(dy/) < cE / e-'^^ly/ - Y^e \ ds 
(4.48) 

< C3e-^^' < C3£. 
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Next, by (4.41), (4.11) and (2.8), 

e > E / e-^"(£(X,) ds + h{dYs)) 

JlT^,T^+l) 

> e-^(^^+i)E r ^\ce,i\Xsr " Ci,2)ds 

> C4e-^^' [E|Xt. r' - C5(l + E{\B\ir')] 
= C4e-^^'[E|XT.r* -ce], 

where €4,05, cg > are independent of e and Y. Hence, by (4.41), 

(4.49) e-^'^'ElXTe r<- + ^e"^^' < c-rS. 

C4 C4 

By (4.45), (4.47), (4.48) and (4.49), 

J(x,y^)<E / e-^^(£(Xf)ds + /i(dy/)) + C8e-^^'(E|XT.r* + l) + C3e 

J[0,T=] 

< J{x,Y) + cge. 
This proves the lemma. □ 

5. Solvability. In this section we establish part (ii) of Theorem 2.1 by 
proving the following result. 

Theorem 5.1. Both V and V are constrained viscosity solutions of 
(2.14) on X. 

A key to the proof will be the following DPPs, Propositions 5.1 and 5.2, 
whose proof, along with the proof of Lemma 5.1, are postponed to Section 8. 
The first DPP regards V. 

Proposition 5.1. Let x eX and e > be given. For Y G A{x) and X 
the controlled process associated with x and Y , let 

(5.1) TY=T = mi{t>0:X{t)(^Be{x)}. 

Then for t € [0, oo), 



V(x) = inf E 

(5.2) 



e-'^'{£{X{s))ds + h{dY{s))) 

[O.iAr] 



+ e-^(*^"V(X(t Ar)) 



In order to present the DPP associated with V, we need to introduce some 
notation. Let $ = {Q,T,{J^t),'^,B) be a system and ( be an M'^ valued 
random variable with probability distribution fi given on {Q,J^q,¥). We 



24 R. ATAR AND A. BUDHIRAJA 

will refer to (^,C) ^ extended system. An J^t adapted RCLL process 
Y with increments in Y is said to be an admissible control for ($,C) if 
Xi = ( + Bt + GYt satisfies Xt € X, P a.s., for all t > 0. Denote the class of 
all such admissible controls by A{^,C)- Let 

(5.3) J{<S>,C,Y) = e[ e-'^'[iiXs)ds + h{dYs)]. 

J[0,oo) 

Denote the class of all J^t adapted RCLL processes with increments in Y by 
^o(^)- Given Y^ G Ao{^) and ( as above, let 

(5.4) Zt = C + Bt + GY,^, rj = f{Z) and y = yO + yor?. 

Then, Y G A{^,C) ^iid the corresponding controlled process X is given as 
Xt = C + Bt + GYt'> + GyoVt- Let 

(5.5) ^t=cT{C,Xs,ris:0<s<t}. 
Let 

A{^, C) = {y € A{<^, C)| there exists Y° G Ao{<^) satisfying (5.4) 

(5.6) ^ 

and y is J-t adapted}. 

Elements of A{^,C) will be referred to as feedback controls. When fi = 5x, 
for some x G X, we will write A{^,C) A{^,x). Let, for x G X, 

(5.7) V{x)=mi inf J(x,$,y). 

The following lemma proves the equality of V and V. 
Lemma 5.1. For all x eli,V{x) = V{x). 

We can now state the second dynamic programming principle. 

Proposition 5.2. Let x G X and e > be given. For Y G A{^,x), let 
Ty be defined via (5.1). Then for t G (0,cx3), 



e-^'{£{X{s))ds + h{dY{s))) 

[0,tAT] 



V{x) = inf inf E 

(5.8) 

-^(*^^V(X(t Ar)) 



e 



Remark 5.1. When using Propositions 5.1 and 5.2 we can assume with- 
out loss of generality, that the infimum is taken only on those Y G A{x) [resp. 
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^(<^, x)] for which, on the set {r < t}, X(t) G dBfr{x). More precisely, in case 
of Proposition 5.1, for t € (0,oo), let 

Ai,t{x) = {y e A{x) : On the set {r < t}, X{t) e dBeix)}. 

Then 

e-^^(£(X(s)) ds + /i(dy(s))) +e-^(*^"V(X(Mr)) 



V(x) = inf E 
YeAi,t{^) 



[O.tAT 



Indeed, for a general Y £ A{x), consider {X,Y) that agree with {X,Y) on 
[0,r) and, on {r < t}, satisfy AY{t) = aAy(r), where a € (0,1] is such 
that X{t) € dB^{x). Denoting by Ji{Y) the expectation on the right-hand 
side of (5.2) and writing 5 = AY{t) - AY{t), clearly, Ji{Y) - Ji{Y) = 
E{lr<te~^[h{6) + y(X(r) + G6) - v[x{t))]} > by Lemma 4.3. A similar 
statement holds for the dynamic programming principle in Proposition 5.2. 

The proofs of the two dynamic programming principles and Lemma 5.1 
are deferred to Section 8. 

Proposition 5.3. Both V and V are viscosity supers olutions of (2.14) 
on X. 

Proof. We will first consider V. Fix x G X and let ip G C^(X) be such 
that V — has a global minimum at x. We can assume, without loss of 
generality, that V{x) — (p{x) = 0. We need to show that either 

(5.9) (5ip{x)+Cip{x)-i{x)>0 
or 

(5.10) inf{Gy • Dip{x) + h{y) : y € Yi} < 0. 

Arguing by contradiction, assume that neither of the above assertions is 
true. Then one can find ^ > and e > such that, for all x G -Be(x) PI X, 

(5.11) pip{x) + Cip{x) - e{x) < -9 

and Gy ■ Dip{x) + h{y) > 9 for all y G Yi. The latter implies that, for all 
yGY, 

(5.12) GyDip{x) + h{y)>9\Gy\. 

Let t > 0, fix y G Ai^tix) and denote U = GY . Let X be the corresponding 
controlled process. Denote 

y=(t) = y(t)- Ay(s), C/= = Gy^ 

0<s<t 
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Then is continuous, C/'^(0) = 0, and it has increments in U. Let r be as 
in (5.1). An apphcation of Ito's formula gives 

/•t/\T 

Jo 

(5.13) -E f e-f^'Dip{Xs-)-dU^ 

J[0,tAT] 
0<s<tAT 

From (5.12), it follows that, for < s < t A r, 

ifiXs) - ^{Xs^) = D^{Xs^ + aAUs) ■ GAYs da 

(5.14) 

>e\AUs\-hiAYs). 

We also have, by (5.12), 

/ e-'''Dip{Xs-)-dU^ 

J[0,tAr] 

(5.15) 

>9 [ e~l^'d\U%- I e-f^'h{dY^). 



[O.tAr] J[0,iA- 

Combining (5.11), (5.14), (5.14), (5.15) and the second part of Lemma 4.2, 
we obtain 

99(x) <E[e-^(*^^)v9(XjAr)] +E / e~^'[l{Xs)ds + h{dYs)] 

J[0,tAr] 

-OE [ e-f^'ids + dpl). 
Since if <V and \Ur\ < /[q ^.j c?|i^|s, we have 

V9(x) <E[e-^(*^^V(XtAr)] +E / e-'^'[l{Xs)ds + h{dYs)] 

J[0,t/\T] 

(5.16) 

-ee-^^E[t^T + \Ut^r\]■ 

Taking infimum over all Y E Ai^t{x), we have from Proposition 5.1 and 
Remark 5.1 that 

V{x) = <f{x) < V{x) - ee-^^a{t), 

where 

a{t)= inf E[t^T + \Ut^r\]■ 
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A contradiction will be obtained by showing 

(5.17) 3t>0 s.t. a{t)>0. 

Recall that X = x + U + B, U = GY and that, since Y G Ai^ti^), one has 
\XtAT — x| = e on {r < t}. Hence, 

E\UtAr\tr<t>me - \BtAr\)tr<t]. 

Thus, 

E[t A r + \UtAr\] > ntlr>t + ie/2)l^^t,\B\;<e/2] >[t/\ {e/2)]P{\B\: < e/2). 

Clearly, for ah i > small enough, P{\B\^ < e/2) > 0. Note that the qualifier 
"small enough" is needed since the Brownian motion is allowed to be degen- 
erate. This proves (5.17) and hence the first part of the result. The proof 
that y is a supersolution as well is similar; instead of taking infimum over 
all Y € Ai^tix) in (5.16), we take infimum over all feedback controls and use 
the dynamic programming principle in Proposition 5.2. □ 

Proposition 5.4. Both V and V are viscosity suhsolutions of (2.14) 
on X". 

Proof. Once more we only prove the result for V. Fix x € X° and let 

If £ C'^(K°) be such that V — if has a global maximum at x. We need to 
show that 

(5.18) f3V{x) + Cipix)-e{x)<0 
and 

(5.19) GyDip{x) + h{y)>0, y gY^. 

We can assume without loss of generality that (p{x) = V{x). Thus, V <(p on 
X°. For all (5 > small enough, one has x + 6Gy G X° for all y € Yi. Hence, 
by Lemma 4.3, 

^{x + 5Gy) - <f{x) > V{x + 5Gy) - V{x) > -5h{y), y G Yi. 

Dividing by 5 and taking (5 — s- proves (5.19). 

To prove (5.18), let e > be such that Bi,{x) C S° . For a control Y and a 
corresponding controlled process X, let 

T^ = lMTii{t:X{t)iBe{x)}. 

Now set = for s G [0,ry] and, thus, Xg = x + Bg for s < Ty. In what 
follows denote t = Ty- An application of Ito's formula gives 

(5.20) V{x) = ip{x) = E[e-f^^ip{Xr)] +E T e-^'{Cip{Xs) + /3(/?(X,)) ds. 

Jo 



28 R. ATAR AND A. BUDHIRAJA 

Using Proposition 5.1, the inequality V <ip and (5.20), 

V{x) <e( r e-f^'liXs) ds + e-f^^ip{Xr)) 

(5.21) - Wo ^ ; 

= ip{x)+K [ e-'^'{l{Xs)-Cip{Xs)-(5ip{Xs))ds. 
Jo 

Recalling that V{x) = ip{x) and denoting 

C{x)=e{x)-C^{x)-(3^{x), xeBeix), 
we have E /J" e-f^^iXs) ds > 0. Hence, 

C{x)E r e-f^'dsy-E r e-f^'{CiX,)-C{x))ds>-aix,e)E re-f^'ds, 
Jo Jo Jo 



where 



a{x,e)= ni_ax \(:^{x) — ({x)\. 



Since r > a.s., it follows that ((x) > —a{x,e). Taking e — > 0, we obtain 
C{x) > 0, proving (5.18) and hence the first part of the result. The second 
part once more is obtained upon using Proposition 5.2 rather than Propo- 
sition 5.1 in proving the statement analogous to (5.22). □ 

Combining Propositions 5.3 and 5.4, we obtain Theorem 5.1. 

6. Uniqueness on bounded domain. Recall the notation X^, 5^, cr = a{r), 
as well as Jr,(p, Vr^cf, and Vr^cf, from Section 4. In particular, recall that 

VrAx)= inf JrAx,Y), VrAx)= iiif JrAx,Y). 

Let r > be fixed throughout this section. In this section we prove the 
following. 

Proposition 6.1. Let u G C+(X) he a constrained viscosity solution 
of (2.14) on X. Assume that (2.10) holds. Then for x ^^r, 

(6.1) u{x) = Vr,u{x) = Vr,u{x). 

Recall Definition 2.2 of viscosity sub- and supersolutions. 

Definition 6.1. Let — >M be given. We say that a continuous 
function '(/' : X^ — > R_|_ is a constrained viscosity solution of (2.14) on X,. with 
the Dirichlet boundary condition (f) if the following conditions hold: 

(i) V is a viscosity supersolution of (2.14) on X^; 

(ii) ip \s a, viscosity subsolution of (2.14) on X°; 
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(iii) tp = (f> on dr ■ 

Remark 6.1. If n : X ^ is a constrained viscosity solution of (2.14) 
on X, then it is clearly a constrained viscosity solution of (2.14) on with 
the Dirichlet boundary condition u\q^. 

The proof of the following result is similar to that of Theorem 5.1 and 
therefore is omitted. 

Theorem 6.1. Let (j)eC+{dr). Assume that (2.10) holds. Then both 
Vr^cf, and Vr,^ are constrained viscosity solutions of (2.14) on X,. with the 
Dirichlet boundary condition (p. 

The following will be the principal tool in proving Proposition 6.1. 

Theorem 6.2. Let cj) G C+{dr). Let u,v G C+(Xr) be two constrained 
viscosity solutions of (2.14) on X^. with the Dirichlet boundary condition (j). 
Then u = v. 

Proof of Proposition 6.1. The function Vr^u is clearly nonnegative, 
and by Lemma 4.6 and Theorem 6.1, it is a C+(Xj.) constrained viscosity 
solution of (2.14) on X^ with the Dirichlet boundary condition u\q^. In view 
of Remark 6.1, so is u. A similar statement holds for Vr^u- Hence, the result 
follows from Theorem 6.2. □ 

In the rest of this section we prove Theorem 6.2. 

Lemma 6.1. For every ^ G X^, there exist rj = rj((^) G R" and a = a(^) > 
such that 

(6.2) Bta{x + tr]) CX° ^ x eX-Ci BaiO and \ft e (0,1]. 

Proof. Fix ^ G X^. Let w G X° and let a > be so smah that B2a{w) C 
X°. If x G Xr and y G X°, then convexity of X^ implies that any point on the 
line segment joining x and ?/, excluding x, belongs to the interior X°. Hence, 
for any x G X^ , 

U B2ta{x + t{w - x)) = {x + t{z-x):z£ B2a{w),t G (0, 1]} C X°. 

te{o,i] 

The inclusion above may be written as 

(6.3) y B2ta{x + t{w-x))cX"^. 
te(o,i] 
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Let r] = w-(. Clearly, for x € Ba{Or\^r, [Jte{o,i]^ta{x + tr]) C [jte{o,i]B2ta{x + 
t{w — x)). Hence, the result follows from (6.3). □ 

Proof of Theorem 6.2. We introduce some notation specific to the 
proof. For n G N, denote by §(n) the space of symmetric nxn matrices. We 
write Ml < M2 if and only if M2 — Mi is nonnegative definite. Mi , M2 € S(?^) • 
Let S be relatively open in X. For X G X, r G R, p G M'^ and A G S(/c), write 

F{x, r,p,A)=(3r-b-p- itrace(i;^) - i{x). 

For rc G 5" and a real valued continuous function -0 on X,., denote 

j|'+0(x) = {{Dip{x),D^ip{x)) : G C^{S) and 

ip — if has a local maximum at x}, 

= {{D^{x),D'^ip{x)) : if G C^{S) and 

tp — f has a local minimum at x}. 

Define the closures of the above sets in the following way. For x (z S, 

il}{x) = {(p, M) G M X %{k) : there exists a sequence 

(x„,p„,M„) gX^ X M'^ X %{k) s.t. 
{pn,Mn) G j|'"^0(x„), and 

),Pn,Mn) {x,lp{x),p,M) 

as n ^ 00}. 

Define J5 ip{x) analogously. For short, write J^'"*" for J^'J^ and similarly 

define J^'~, J^'^ and J^' . Generic elements of Y will be denoted by y 
(rather than y). 

To prove the result, it suffices to show that 

(6.4) u{x)<v{x), x£Xr. 

Let ci = Oq ^. Note that ciGy ■ ni > 1, ?/ G Yi [cf. (2.2)]. Let '0(x) = cix ■ ui — 
C2 , X G Xr , where C2 is fixed and large enough so that 

(6.5) i'ix)<-l and (£ + /?)V'(x) - ^(x) < -1, xGX^. 
Thus, 

(6.6) Gy-Dip{x)>l, xGX^,yGYi. 
For a G (0, 1), define 

(6.7) Ua = oiu + {1 — a)il) . 
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It suffices to show that, for every a G (0, 1), 

(6.8) Ua[x)<v(x\ XGXr. 

We argue by contradiction and assume that (6.8) does not hold. Therefore, 
there exist a S (0,1) and ^ € such that 

(6.9) UaiO — = niax(tiQ,(2;) — v{x)) = 6 >0. 

We first argue that if z S X° and (p, ^) G J ' Ua{z), then 

(6.10) F{z,u^iz),p,A)<-{l-a), n{p)<-{l-a). 

To this end, consider first (p, A) € J^'''"Uq,(z). Let p* and A* be such that 
p = ap* + (1 — a)Dilj{z)^ 

(6.11) 

A = aA* + a)D'^i){z) = (1 - a)A*. 

Then by (6.7), {p* ,A*) e J'^'~^u{z). Using the subsolution property of u (cf. 
Definition 6.1), we have 

(6.12) F{z,u{z),p*,A*) <0. 

Noting that the map (r,q,X) F(z,r,q,X) is affine and combining (6.5), 
(6.7), (6.11) and (6.12), we obtain the first inequality in (6.10). Now since 
z € X° and {p,A) G j'^'~^Ua{z) are arbitrary, and F is continuous in all 
variables, the first inequality in (6.10) holds, in fact, for all z € X° and 

{p,A) E J ' Ua{z). 

By convexity of 7i, (6.6) and the subsolution property of u, it is seen that 
n{p) < an{p*) + (1 - a)n{DiP{z)) < -(1 - a), 

2 -f 

for all {p,A) G J ' Ua{z). This proves the second inequality in (6.10). 

Recall that is defined via (6.9). By (6.5) < u on X,., and since u = 
u = on (9r, it is impossible that ^ ^ dr- Let i] = r][^) be as in Lemma 6.1. 
For 7 € (1, oo) and e G (0, 1), set 

■^{x,y) = |7(j;-?/) -e7?p +e|y-Cp, 

$(x, y) = Ua{x) - v{y) - ^{x, y), {x, y) G X,. x X^, 

and let 

{xen^yeri) = i^^y) ^ argmax ^{x,y). 

By Lemma 6.1, 

(6.13) e+-r/GX°. 
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Clearly, <I>(x,y) > ^>(^ + 7^^577, ^). This can be rewritten as 



(6.14) u^{x) - v{y) -Ua\i + -r]j + v{0 > h{x - y) - er]\^ + e\y - 

Dividing by 7^, we see that, for every e, |ir — y|— >0as7^oo. This obser- 
vation, along with (6.9), (6.14) and the continuity of and v, gives that 
limsup^^j^ |7(x — y)— eri\^ + e\y — < 0. Hence, for all e € (0, 1) 

(6.15) y —* l{x — y)^srj as 7 — > 00. 

In particular, 5; = y + 7~^er/ + 7~^o(l), as 7—^00. Hence, by (6.13) and 
Lemma 6.1, x G X° for 7 > 70, for some 70 = 7o(e) < 00. By (6.5), (6.7), (6.9) 
and nonnegativity of u, it follows that w(^) < u(^). By choosing 70 larger if 
necessary, we have v{y) < u{y) for 7 > 70. Henceforth, assume 7 > 70. Since 
£, ^ dr, we have that, for large 7, 

(6.16) yeXr. 
For {x,r,p,A) G x M x M'^ x §{k), let 

G{x,r,p,A) = F{x,r,p,A)yn{p). 

Since x G X°, we have from (6.10) that 

G{x,Ua(x),p,X) < -(1 - a), {p,X) G j'^'~^Ua{x). 

By (6.16) and the supersolution property of v, 

G{y,v{y),q,Y)>Q V (g, F) G j''"?;(y). 

Combining the above two displays and using the inequality (aV6) — (c Vd) < 
(a — c) V (6 — d), {a,b,c,d) G M^, we obtain that, for all {p,X) G J ' Ua{x) 
and {q,Y) G J ' v{y), 

1 - a < G{y,v{y),q,Y) - G{x,Ua{x),p,X) 

(6.17) < max{|g — p\, P{v{y) — Uaix)) + \b\\q — p\ 

+ \i{x) - liy)\ + (l/2)trace(S(X - ¥))}. 
Next, noting that Uaix) — v{y) > ^{x,y) > ^{£,,0^ using (6.9), we have 

(6.18) v{y) -Ua{x) <e^\T]\^. 
Hence, by (6.17), 

1 — a < max{|g — p|, /5e^|ryp + |^| 1^ — p| 

(6.19) 

+ \i{x) - e{y)\ + (l/2)trace(S(X - Y))}. 
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By Theorem 3.2 of [8], for each a € (0,oo), one can find X,Y ^ S(/c) such 
that 

and 

(6.20) ^y^<D''-^{x,y) + a{DH{x,y)f. 

Observe that 

Dx'^{x,y) = 27(7(5 -y) -er/); 

(6-21) 

-Dy^{x,y) = 2jijix - y) - er]) - 2e{y - 

(6.22) D'^{x,y) = 2^'^^^ "/)+2^(o 5) " 
From (6.20), it follows that, with 1 = (/,/)', 

X-Y = l'(^^ I'iD^^ix, y) + a{DH{x, y)f)l. 

This implies that 

trace(S(X - Y)) < trace{T,l'{D'^^{x, y) + (t{D'^^{x, y)f)l). 

From (6.22), it now follows that, for some <; G (0, 00) which is independent 
of e,7,cr, 

(6.23) (l/2)trace(E(X - Y)) < qe. 
Using (6.18), (6.21) and (6.23) in (6.20), we have 

1 - a < I3e^\i]\^ + 2e\y-i\{\h\ + 1) + \l{x) - l{y)\ + ?e. 

Letting 7 00, using (6.15) and continuity of i, we obtain 

l-a</3e^|r/p + ?e. 

Finally, letting e ^ we arrive at a contradiction. Hence, (6.8) must hold, 
and the result follows. □ 

7. Uniqueness on unbounded domain. This section proves uniqueness of 
solutions to (2.14) on X. The two results stated below establish part (iii) of 
Theorem 2.1. 

Theorem 7.1. Suppose that (2.10) holds. Let u G Cpoi,+ (X) be a con- 
strained viscosity solution of (2.14) on X. Then u <V {in particular, u < 
V). 
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Theorem 7.2. Suppose that (2.10) holds. Let u G Cpoi,+ (X) be a con- 
strained viscosity solution of (2.14) on X. Then u>V provided that, in 
addition, one of the following holds: (a) u € Cp^j ^_ (X) ; or (b) (2.11) holds. 

While both results above hold regardless of the value of mi € [0,oo), in 
the special case where mi = 0, the statement of part (a) of Theorem 7.2 is 
void: If a solution u belongs to Cpoi,+, then by Theorem 7.1, u<V , and by 
Lemma 4.4, V is bounded, and so u cannot lie in Cp^j 

Before proving the above, we show that these results, along with the 
results of the previous sections, imply Theorem 2.1. 

Proof of Theorem 2.1. For part (i), see Remark 4.2. Theorem 5.1 
establishes part (ii) . We now consider part (iii) . Let mi > and u G Cp^j _^ 
be a constrained viscosity solution of (2.14) on X. Then Theorem 7.1 and 
Theorem 7.2(a) establish that V = u and so by part (i) of the theorem, we 
get that V is the only solution in the class C^^y^- For the case mi = 0, 
Theorem 7.1 establishes the maximality of V among solutions in Cpoi,+ . 
Finally, under (2.11), uniqueness in Cpoi,+ follows from Theorem 7.1 and 
Theorem 7.2(b). This proves part (iii). 

The identity V = V under condition (2.10) is an immediate consequence 
of parts (i)-(iii). □ 

Proof of Theorem 7.1. Fix u and let Cu,mu be such that u{x) < 
Cn(l + Ixl'""), X € X. Recall the notation of Lemmas 4.7 and 4.8. Fix x G 
X, a system y e Apix) and e G (0,1). Let T^y^X^Z^ be defined 
via (4.41), (4.42) and (4.43). Let a = a{r) be as in (4.2). Now we esti- 
mate E[e-^'^(|X^p" + 1)]. Recalling the definition of Af{^,x) and (4.12), 
we have that, for every t, 

P{a <t) = P{\ui ■ X\l >r)< 2ao V~^E(|Xtj + 1^1^*) ^ 0, 
as r ^ oo. Hence, a{r) oo a.s. as r ^ oo. By (4.12), 

W.e-^'^dX^r- + 1) < Co W^[(|X|,.| + \B\*T.r- + 1]. 
Since Y G Api^jx), {\X^e \ + |i?|^s)™" is integrable and, thus, 
(7.1) lim E[W^e-^"(|^^r" + 1)] = 0. 

Let g{e) = gvie) = e-^^'E|X(r^)p" . Once more, since Y G Af, and 
oo as e ^ 0, we have g{e) — > as e — > 0. Hence, by (4.41) and (4.44), 

E[W^e-^'^(|X^r" + l)] 

< ci{e-^^\E\XTer^ + 1) +E(l<,>T^^e-^^|X^ - X^.T")) 

<C2{e + gY{e)+E[l^yT^e-^''snp{\Bs-BT^r-:se[T',a]}]) 
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C2ie + gvie) + ^K[l„^lT'^+i,T^+i+i)e' 

\ i=0 



-l3{T^+i) 



sup{|s, - ^tH""" : s e [^^^^ + i + 1]}] 



< Ci{e + gY{e)). 
Combining this with (7.1) we have 

(7.2) hmsupE[e^^'^(|X^r" + 1)] <C5(e + 9y(e)). 

r— ♦oo 

Next note that the control need not be in Ar, but we can modify it 
as foUows. On {aoo} let X^'^ = . On {a < oo}, let {X^''{t),Y^''-{t)) = 
{X^{t),Y%t)) for tG [0,cj) and, recalling (4.17), set 

X'^'ia) = X'{a-) + jAX'ia), Y'^'{a) = Y%a-) + j^Y^a), 

where 

j = jr.{X'{a-),AX%a)). 

Then X^'"^ = x + B + GY'^''' on [0, a] and moreover, Y^'"" G A- Since 7 < 1, 
/i(Ay^'^((7)) < h{AY%a)), and by (2.2), |X"''-(f7)| < a^^ui ■ X^'^{a) < 
aQ^\X'^{a)\. It therefore follows from Proposition 6.1 that 



u{x) < E 
< E 



/ e-f^'ii{Xp^) ds + /i(dy/'^)) + c^e-^'^dX^'H'"" + 1 

J[0,cr] 

/ e-f^'iliXl) ds + /i(dy/)) + c^ao ™"e-^^(|X^r" + 1) 



Thus, by (7.2) and Lemma 4.8, 

u{x) < J($, X, Y') + ce{£ + grie)) < J($, x, Y) + c-jie + gyie)). 

Sending e — > and recalling that Y € ^i?(<&, j;) and x € X are arbitrary, we 
conclude by Lemma 4.7 that u < F on X. □ 



Proof of Theorem 7.2. Once again, let u be fixed and let Cu,mu 
be such that u{x) < c„(l + Ixp"), x G X. Fix x eX. Let e e (0,eo) be 
given, where the constant eo > will be chosen later. We will use the re- 
marks below (2.13) and Proposition 6.1. We will construct a system $ = 
(0,^,P,^f,S) and Y £ A{^,x) such that u{x) > J{x,Y) - ce with a con- 
stant c not depending on e. This will clearly yield the result. 
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Consider first the case where u € Cp^j _,_ (X) , namely, u has compact level 
sets. Then Ru{r) = min^^ u satisfies Ru{r) ^ oo as r — > oo. Let p € (0, oo) 
and ^ G Xp. Consider the minimization problem associated with the right- 
hand side of (6.1) on X^, where r > p. If y € Ar{S,) satisfies Jr^uiC,y) < 
'^r,uiO + 1> then from Proposition 6.1, with a = a{r) [cf. (4.2)], 

(7.3) E[e-^'^n(X„)] +E / e-^'hidYs) < c„(l + 1^1™") + 1, 

where X is the controlled process corresponding to Y. Since G 9^, we 
have 

(7.4) 7i(X,(,))>ii„(r). 
Therefore, by (7.3), 

P(a(r) < 1) < e^i?4r)-i[c„(l + l^r^) + 1]. 
This shows the following: 

For every p, one can find r = r{p) > p 

(7.5) 

such that F{a{r) < 1) < 1/2, G Xp. 
Next consider the case where (2.11) is assumed and u G Cpoi,+. By (2.6) and (2.11), 

/ e-^'hidYs) >Ch [ e-^' d\Y\s > Che-^''\Y^\. 

J[0,cr] J[0,(t] 

Since ui • ^ < p and ui • X„r, we have that 



E 



/ e-f^'h{dYs) > Ch\G\-^Ee-f^''ui ■ GY„ 

JlO.a] 



(7.6) >Ch\G\-^ne-''^r-p-ui-B^)] 

> Ch\G\-'{r - /5)Ee-^'^ - ChlGl-'ne'f'^lBll], 
where |G| > denotes the operator norm of G. Note that 

oo 

ne-'^lBi:] < ^ l,e[,,,+i)e-^*ci(z + 1) < C2, 

i=0 

where C2 does not depend on Y and r. Thus, 

E / e-^'h{dY,)>Ch\G\~\{r-p)Ee-^'' -C2]. 

J[0,a] 

Combining this with (7.3), we have that (7.5) holds in this case as well. 

Define inductively a sequence of domains X^^ , Xp„ , ui ■ x = pQ < ri < pi < 
r2 < • • ■ via the relations r„ = r{pn-i), n G N, and pn = Vn + I, n G iKl. For 
n G Ihi and 6 >0, denote 

mn{6) = max{M(^) - u{z) i^, z eXp^,\^ - z\ <6}. 
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By assumption, uq S 1LJ° [cf. the comment following (2.1)]. Hence, there is a 
constant ai > such that Baj^{uo) C U" fl X°. Fix such ai. Fix also k lin- 
early independent unit vectors € i?ai (uq) such that, with A = cone{ui,i = 
1, . . . , fc), one has uq G A°. Thus, there exists a > such that Ba{uo) C A. 
Let such a be fixed and denote C3 = (1 + a~^)A;^/^. Let ^ : A — > Y be the 
linear map of Lemma A. 2 and denote by \q\ its operator norm. For n G N, 
let &n denote the finite set 

6„ = (An,,Z^)nXp„, 

where Xn^e > are fixed constants, so small that, for n G N, 

(7.7) (1 + |^|C3)A„,, <e2-" and m„(c3A„,,) < £2"". 
We next show that 

(7.8) yC(^dr„3z£6n S.t. Z-eeA, \z - C\ < C3Xn,e- 

Indeed, given € (?r„ we have + C C X, where C C A denotes the cone 
generated by Ba{uQ). Since C can be written as Ua>o -^aa('^^o)) we have 

It is easy to see that, for every w ^M^, there exists z € Xn^e'^^ such that 
I2; — < fc^/^A„,^e. This shows that Z contains a point in Xn.sTl^ ■ Choosing 
eo = C3 ^, we have — C| < cj,Xn,e < 1 for every z^Z and e G (0,eo), and thus 
for all such Z d X^^+i. Thus, Z contains a point z G 6„. By construction, 
2; — ^ G A and, therefore, (7.8) holds. 

For n G N, let M„ : dr„ —<■ S„ denote a measurable map such that, for 
every ^ G 9r„, condition (7.8) is met by z = M„(^). By Lemma A. 2, 

(7.9) GgiMn{0-0 = Mn{0-C- 

Consider a complete probability space ($7,.?^, P) supporting countably 
many independent (b, S) Brownian motions. In particular, let a (5, E) Brow- 
nian motion B"^'^ be associated with each n G N and z G and let a (6, S) 
Brownian motion B^'^ be associated with x. For each z G &n, consider the 
minimization problem associated with the right-hand side of (6.1), substitut- 
ing Vn+i for r and z for x. We write the system {Q,J^,J^''^,F,B"''^), where 
J-^'^ is the P completion of the cj-field generated by B"^'^, as is 
defined similarly. 

Using Proposition 6.1, find a y"'^ g (-z, ^"'^)) for which 



E 

(7.10) 



/ e-^*(£(X"'") + KdY,'''')) + e-'3-('^"+i)tz(X"; 

<'u(z) +e2-", 
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where X"'^ is the controhed process corresponding to y"'^. To account for 
the dependence of a{rn+i) on the initial point z G 6^) it will be more conve- 
nient to write in what follows cj"'^ for a{rn-^-i). By construction and by (7.5), 
for each pair {n,z), 

(7.11) P(ct"'^ < 1) < 1/2. 

Define inductively a sequence of processes {Bn,Yn,Xn) as follows. Let 
Bi = and ai = 0"°'^. Let Hi = X°'^(a°'^) and note that Hi e dr,. Let 
also 

(7.12) Zi = Mi(Hi), Yi = yO'"l[o,,o,.] + £^(Zi-Hi)l[,o,.,oo), 

and Xi = x + Bi + GYi on [0, (Ti] [we do not define Xi on {ai , oo)] . Note that 
by (7.9) and (7.12), Xi{ai) = Zi, so by (7.8), \Xi{ai) -Ei\< csAi,^. Also 
note that AYi{di) = AY^''^{ai) + Q{Zi-Ei). Hence, by (4.6), (7.7) and (7.10) 



(7.13) 



E / e~'^\i{Xi{t))dt + h{dYi{t))) + e~^''^u{Xi{ai)) 

.J[0,ai] 

< u{x) + e2"-^ + \h\ |£'|c3Ai,£ + mi(c3Ai,e) 

< u{x) +C4e2"^ 

Consider n > 2. On the set {dn-i = oo}, let dn = oo and (fi„,y„,X„) = 
(i?„_i, y„_i, X„„i). Next consider the set {Bn-i < oo}. Let 

B„ = B„„ilp^^_^] + (i^-l.^-- - i?"-l'^"-(?n-l))l(-_,o,) 

and CT„ = CT„_i + fj"-^'^"-i. Let E;„ = X"~i'^"-i(cr"-i'^"-i), and 

Let Xn = X + Bn + Yn on [0,CT„] [and we have not defined it on (an,oo)]. 
By (7.8), \Xn{dn) — S„| < c^\n,e- Denoting the filtration generated by B^ as 
J^", we have from (7.10), in a manner similar to the proof of (7.13), that 

/ ^ e-^\i{Xn{t)) dt + KdYnim + e-^^"n(X„(a„))|.^^^ ^ 

<u{Xn{dn-l))+Ci£2-'', 

a.s. Iterating the above inequality and using (7.13), we now have 
e[ / e-^\l{Xn{t))dt + h{dYn(t))) + e-^^"n(X„(a„)) 

n 

(7.14) <u(x) +C4^e2-" 

< u{x) + c^e. 



E 
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To see that a„ ^ oo a.s., let Fq = {0, Q}, and for n € N, let F„ be the sigma- 
field generated by and {Bn{s),Yn{s) : s € [0, oo)). Then a„ — an-i € Fn 
and, by (7.11), P(a„ - > l|F„_i) > 1/2. By the second Borel-Cantelh 
lemma (cf. [13], page 240), an— ^ oo a.s. 

Since an ^ oo a.s., the limits B = lim„i?„, Y = lim„l^ are well defined 
outside a null set. Let X = x + GY + B. By construction, the process B is 
a (6, S)-Brownian motion, and Y S A{x,^,B). Finally, by (7.14), J{x^Y) < 
u{x) + c^e. Hence, V{x) < u{x) + c^e, and the result follows since e > is 
arbitrarily. □ 

8. Dynamic programming principles. In this section we prove Proposi- 
tions 5.1, 5.2 and Lemma 5.1. 

Proof of Proposition 5.1. Fix t e (0, oo), x € X and Y g A{x). For 
brevity, we denote t At by 9. Note that 



E 



[6»,oo) 



e-f^%iiXs)ds + hidY}))\Tg 



>e-^V(Xe), 



where 1"^^ = Ysls<ce + {Yg — AYq)1s>0. The proof of (8.1) follows in a straight- 
forward manner on recalling that J^t is generated by W and using the strong 
Markov property of W. This immediately shows that 



j(x,y) >E 



/ e-^'{e{Xs) ds + h{dYs)) + e-f^'mAYe) + V{Xe)) 
-J[o,0) 

Taking infimum over Y € A{x) in the above inequality, we have that 

/ e-^'{i{Xs) ds + h{dYs)) + e-f'iViXe)) 



i.2) V(x) > inf E 
YeA{x) 



Now we prove the reverse inequality. Once again, fix x G X and Y S A{x). 
Let 6 G (0, oo) be arbitrary. Then 



3Y £ A{x) and the corresponding X s.t. 
Y{s)=Y{s),se[0,e) and (8.4) holds: 

/ e-^'iliX,)ds + h{dYs)) - e-^%{AYe)\Te 
J[e,oo) 

The proof of (8.3) is provided in the Appendix. This shows that 
E 



i.3) 



iA) e-^^V{Xg)>E 



6. 



f e-^'{e{Xs) ds + h{dYs)) + e-f^'iViXe)) 
e-^\l{Xs)ds + h[dYs))-5 



> E 

'[0,oo) 

> Vix) - 6. 
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Taking infimum over all Y G A{x), the above inequality and (8.2) establish 
the result. □ 



Proof of Proposition 5.2. The proof is adapted from that of Propo- 
sition III. 1.1 of [4]. In view of Lemma 5.1, it suffices to work with V. Let ^ 
be a system, be a J^o measurable M!^ valued random variable with probabil- 
ity distribution fJ-,Y ^ C) and Xt = C + Bt + GYt be the corresponding 
controlled process. From arguments in Theorem 1.1.6 of [4], it follows that 
the conditional law of X given (" = x is /i-almost surely the law of a con- 
trolled process X^ corresponding to some Y^ G A{^^,x) and some system 
Furthermore, 



J J{x,<^^,Y-)^x{dx) = J{^,C,Y). 



Also, following Lemma III. 1.1 of [4], we have that, given e > 0, there exists 
an extended system ($,C) and Y G A{^,C) such that Y^ defined as above 
is e-optimal for V{x), for fi-a.e. x. That is 

J{x,^'',Y'') <V{x) +e, /i- a.e. X. 

We call such a y an e-optimal control for (<&,C)- Now fix x G X. Let Y^ G 
A{^, x) and let X^ be the corresponding controlled process and r = ryi be 
defined via (5.1). Once more we will denote tArhyO. Let n now denote the 
probability distribution of X^{6) and let Y G A{^, C) be an e-optimal control 
given on some extended system (^>,C)) with probability law of C equal to /i. 
Let X be the corresponding controlled process. By augmenting $ suitably, 
one can construct on it processes X and Y such that Y G A{^,x), X = 
x + B + GY, X^{s)=X{s), Y^{s) = Y{s) for sG [0,^) and the conditional 
distribution of {X{6 + -),Y(9 + -) — Y^{9)) given Tg is, for almost every to, the 
same as the distribution of {X^{-),Y^(-)), with z = X^ (9 {ui) , uj) . Here J-f = 
a{X^(s),Y^{s),s < t}. For details on this construction, we refer the reader 
to the proof of Theorem III.l.l of [4]. Next, setting Y{s) = Y{s + 9) -Y^{9), 
we have 



V(x) <E 
<E 



e-'^'{i{Xs)ds + h{dYs))+ / e-f^'{£{Xs)ds + h{dYs)) 
[0,0] J{e,oo) 



m 



e-f^'iliXl) ds + h{dY})) + e~^^h{Y{0)) 



+ E 




\f ^ 






y(o,oo) 



e-f''{e{Xe+s)ds + h{dYs))\Te 



E / e-f^^ml) ds + h{dY})) + E[e-^V(x, y^)U=xi(e)] 

J [0,9] 
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<E 



J[0,9] 



where the second inequahty follows on observing that h^AYg) < h^AY^) + 
/i(y(0)). Taking infimum over all Y^ G A{^,x) and over all systems $ and 
letting e ^ 0, we obtain 



V{x) < inf inf E 



[0,0] 



e-'^'{iiXl)ds + hidY})) 



Next, let <I>^ be a system and Y^ G A{^^,x) be e-optimal for V{x). Then 



(8.6) 



V{x)+e>¥. e~^'{l{Xl)ds + h{dY})) 

+ E f e-f^'{£{Xl)ds + h{dY})). 
J{e,oo) 



Once more, the arguments in Theorem I. 1.6 of [4] yield that the probability 
law of {X^{e + -),Y^{e + -)-Y^{e)) given is, for almost all w, the law of 
{Xl,Y^), where Y^ E ^(<I)^,x), AY^{Qi) = a.s., is some system, Xl is 
the corresponding controlled process and x = X^{9{<jj),uj). This shows that 



E 

Thus 



(e,oo) 



e~^'{£{Xl)ds + h{dY}))\Te 



>e~^''J{x,^\Y^)\,^x^e{.),uy 



E / e~^'{l{X]) ds + h{dY})) > Kie^l^^ViXl)]. 
J{e,oo) 

Substituting the above in (8.7) and taking e ^ yields 



V(x) > inf inf 
<J> ^ 



E 



m 



e-'^'i£{Xl)ds + h{dY})) 



The result follows on combining (8.5) and (8.7). □ 



Proof of Lemma 5.1. Fix x G X, a system ^ and Y G A{^,x) such 
that J(x, <l>, Y) < oo. In view of Lemmas 4.7 and 4.8, we can assume without 
loss of generality [cf. (4.44)] that 



e"^*E|GyJ^O ast^ooandE 



-/3t 



[0,oo) 



GYAdt <oo. 
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Write Yt = Yf + Yf, where Yf = Eo<s<t and Yf = Yt- Yf. Note that 
both Y'^ and Y'^ are RCLL, Tt adapted and have increments in Y. Fur- 
thermore, the measures d\Y'^\ and d\Y'^\ are singular and, therefore, from 
Lemma 4.2, 

J($,a;,y) = E / e-^'[l{X,)ds + h{dY^) + h{dY^)l 

J[0,oo) 

where X is the controlled process corresponding to Y . Fix 5 > and define 

= Y^i.2)s +-At-ik- miY^u^s - yik-2)5), 

(8.9) 

te[{k-l)6,k6),k = 2,3,.... 

Set Y^^ = Ig^ = for t G [0,5). Note that Y^ is continuous J^t adapted and 
has increments in Y. Also let 

(8.10) Xf = T{x + B + GY^ + GY'^){t) =x + Bt + GY^ + GYf + Gyo4, 

where t?^ = r(x + + GY^ + GY'^){t). Define = - Gyo-df = x + Bt + 
GYf + Gy/. Note that A^^^ = GJ\Yf = GAJt. Thus, A^^^ = GE(Ayt | A^^^). 
Define = X;o<s<i lE(Ay^|ACf ). Clearly, Y'^ is adapted to the filtration 
(T{Xf , 7?f , s < t}. Furthermore, using the convexity of /i, it is easy to see that 

(8.11) e/ e-f^^hidYf) <^ [ e-f^^hidYf). 

J[0,oo) J[0,oo) 

Also, it is easy to check that GY/ = GYf. Now let li = x + Bt + GY/ = 
x + Bt + /o* Gy/ ds, where y/ = " ^(fc-2)5)l.e[(fc-i)5,fc5) and y/ = 

for s S [0,(5). Note that rjf = — GY^ and, therefore, it is Tt adapted, 
where Tt = cj{Xf ,19^ : s <t}. Thus, from Theorem 4.2 of [33], there exists 
a (6, S) — j^t-Brownian motion Bt such that 

(8.12) 4 = x + Bt + G f\{Y,^\Ts) ds. 

Jo 

Note that Y^ = /q E(y,''|^<j) ds is continuous, .T^t adapted and has increments 
in Y. Once more, using the radial homogeneity and convexity of h, we have 

(8.13) e/ e-^*/i((iy/) < E / e~'^*/i((iy/). 

J[0,oo) J[0,oo) 

Thus, defining y/ = y/ + y/, we have by (8.10) and (8.12) Xf = x + Bt + 

GYf + Gyo^f and therefore, = Yf + yo^f G A{^,x), where $ = (17, .F, (J't),P,S). 

In order to prove the proposition, it now suffices to show that 

(8.14) limsupJ(x,l>,y'^) < J(x,$,y). 
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From (8.10) it follows that, for all T G (0, cx)), 

sup [\X\t)-X{t)\+'d\t)]<ci sup \GY^ [t) - GY%t)\ 

0<t<T 0<t<T 

(8.15) 

<ci sup \Y\t)-Y^{t)\ 

0<t<T 

and due to the sample path continuity of Y'^, the last term approaches as 
5 — > 0. This, along with continuity of i and the Lipschitz continuity of h, 
shows that, for all t G (0, oo), 

£{Xf) ^ £iXt) and sup (^9f + |/i(y/) - /i(y/)|) ^ 

0<s<t 

(8.16) 

a.s. as (5 ^ 0. 

Also, using (2.8) and (2.2), it can be seen that 

i{xi) < C2(i + (|GF|:)™^ + {\GYxr' + {\B\ir') 
<c3{i+iix.,)+{\B\:r'). 

Thus, recalling that E /rg e~^*£{Xs) ds < J{x, Y) < oo, we have from (8.16) 
and an application of dominated convergence theorem that, as 5^0, 

(8.17) e/ e~f^H{Xl)ds^^ [ e~'^H{Xs)ds. 

J[0,oo) J[0,oo) 

Next, combining (8.11) and (8.13), we have 

e/ e-'^'hidYl)<E f e-^'h{dYf)+E f e-^'h{dY^) 

J[0,oo) J[0,oo) J[0,oo) 

+ /i(Gyo)E / e-f^' d-di 

J[0,oo) 

<e/ e-'^'h{dYf) + E f e-f^'h{dY^) 

J[0,cxd) J[0,cxd) 

+ h{Gyo)E [ e-f^'d-dl. 

7[0,oo) 

By (8.9) and (8.15), 

(8.19) 4<ci sup \GY\t)-GY\t)\<2ci sup \GY^{s)\<Ci\GY{t)\, 

0<s<t 0<s<t 

where (2.2) is used in the last inequality. Combining this observation with (8.8), 
we obtain e-^^EiJf ^ as t ^ oo. Thus, E /p^^-, e"^** = ^E /[g^^-, e-P'-dl ds. 
Combining (8.19) and (8.8), we have, by dominated convergence, 

(8.20) E / e"^" d^i^O as 6 ^ 0. 

J[0,oo) 



^.18) 
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Next we show that 

(8.21) e[ e-^'h{dY^)<W. [ e-l^'h{dY^). 



Note that 



(8.22) e/ e-^'h{dY^)=Ey I e-^'h{dY,^). 

•''[0,00) p{Jijs,ij+m 

Also, for j = 1,2,.. ., 

(8.23) / e-^'h{dY^)= [ e-f^'h{Y^) ds < h{Y s), 

■J[js,u+m Jijs,ij+m 

where Yj^g = e~^^^{Y'^(jS) — Y'^{{j — 1)5)) and the last inequality follows 
upon observing that on the interval [j5,{j + 1)5), Y^ equals 6~^{Y'^{j6) — 
Y^{(j — 1)6)). Using convexity and radial homogeneity of h, we see that 



h{Y'^{j6) - y^((j - 1)5)) < / h{Y'n d\Y%, 

where Y'''° = dY^dlY""]. Thus, 

/ e-'^'hidYf) > e~<^^^ f h{dYf) 

>e-''^'h{Y'{j5)-Y'^{{j-l)6)) 
= h(Yj,5). 

Using the above inequality in (8.23) yields that J^j^ (^j_^-^^^g-je~^'^h{dY^) < 
I[{j-i)S j5)^~^^^(d^s)- Combining this observation with (8.22) gives (8.21). 
Using (8.20) and (8.21) in (8.19), we get 



limsupE / e-^'h{dYl)<E f e-f^'h{dYf) 

5^0 JIO.oo) J[0,oo) 

^.24) +E / e-^"/i(dy/) 

ifO.ool 



'[0,oo) 

= E / e-^'h{dYs). 

J[0,oo) 

Combining (8.25) and (8.17), we obtain (8.14) and the result follows. □ 

APPENDIX 

Lemma A.l. Let Yt he an Tt-o,dapted RCLL process with increments in 
Y. One can find an Tt-progressively measurable process Y^ with values in 
Y n 5P~i such that /^q Y° d\Y\s = Yt, t > 0, a.s. 
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Proof. Since Tt satisfies the usual conditions, it is right continuous. 
Moreover, since Y is J^t-adapted and ROLL, it is ^(-progressively measur- 
able (see Proposition 1.13 of [24]). Hence, by Appendix D of [15], there 
exists a progressively measurable process Y° such that \Xt\ — 1' ^ 0, and 
Y^ = /jQ^j y^°(i|y|( a.s. (in [15] Yt is left continuous and the integral is over 
[0,t), but the adaptation is clear). It remains to show that 

(A.l) P(|yi°| = l, (i|y|-a.e.) = l. 

If y is a deterministic ROLL path with increments in Y and I is a finite 
interval of ]R_|_ , 

J^d\y\t = sup^ ytd\y\t <sup^^ \yt\d\y\t = J^\yt\d\y\t, 



k 



k 



where the suprema range over partitions (1^) of I. The above inequality for 
intervals implies the same for Borel sets S of ]R_|_, thus. 



(A.2) / d\y\t< / \y°t\d\y\t. 

Js Js 

Fix T > 0. Given e > 0, (A.2) imphes 

/ llYol<l^ed\Y\< f l|yo|<i„,|y°|d|y| 
J[0,T] J[0,T] 

< (1 -e) / l|yo|<i--£(i|y|, a.s. 

J[0,T] ' ' 

and, therefore, E /jq j,-^ 1\y°\<i-£ d\Y\ = 0. Since limg^o /[o t] '^\Y°\<i-e d\Y\ = 
/[Q l\Y°\<id\Y\ for a.e. w, we have by Fatou's lemma that E j,^ 1\y°\<i d\Y\ 
0, proving (A.l). □ 

For the result below, note that G has full row rank, that is, k, and Y 
has nonempty interior, and therefore, one can find k linearly independent 
elements of U. 



Lemma A.2. Let G U, i = I, . . . ,k, be linearly independent unit vec- 
tors. Let A = cone{ui : i = 1, . . . ,k) . Then there exists a linear map : A — > Y 
such that u = Gq{u) for all n G A. 

Proof. For z = 1, . . . , A;, fix S Y such that Gyi = Uj. Every n G A can 
be written as Y^^=i(^i'^ii where Ui = ai{u) > 0. Also, ai{u) are uniquely 
determined by and depend linearly on n S A. Setting 



k 

g{u) =^ai{u)yi, u G A, 

i=l 
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yields the result. □ 



Proof of claim (8.3) . Prom Lemma 4.5, we know that V is continuous 
and so uniformly continuous on B2e{x) PI X. Thus, there exists a G (0,oo) 
such that 

(A. 3) \V{xi)-V{x2)\<5/'i for anxi,a;2€-B2e(2;)nXs.t. |xi-X2|<?. 

Let the polyhedral cone A C U and the linear map q be as in Lemma A. 2, 
and denote by |^| the operator norm of q. By making <f smaller if necessary, 
we can assume that <i\Q\ < (^/3. Now let Zn = B2e{x) n X n n~^'L^ . Note that 
Zn is a finite set and for n sufficiently large, we can find a measurable map 
•dn '■ Bir{x) Zn such that 

(A.4) ??„(0-^gA and \K{0-^\<^ for alU G ^^(x) n X. 

Pix such n and suppress it from the notation. Por each £, £ Z, find 1^ E -^(0 
such that 

(A.5) V{0>E f e-'^'{e{X^{s))ds + h{dY^{s)))-6/3, 

J[0,oo) 

where is the controlled process corresponding to 1^. Note that for each 
^ G ^, as in the proof of (8.1), we can find measurable function : [0, oo) x 
)^+ ^ Y such that 

F^{s,{B{tiAs)}^>i) = Y^{s), se [0,oo). 

Now define for s > Y* (s) = F^(^x{e)) (s, {BiiU As) + 9)- B{9)}i>i). Pinally, 
define Y^ = Ysls<e + (Y* + le + gi'&iXe) - Xo))ls>g and let X be the cor- 
responding controlled process. By construction and using (4.6), it is easy to 
check that 



E 



/ e-^'{£{Xs)ds + h{dYs)) - e-''%{AYe)\Te 
J[e,oo) 

< e~^^V{Xe) +5. □ 
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